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PREFACE 


The  use  of  competition  in  weapon  system  acquisition  is  widely 
advocated  in  policy  statements  issued  by  the  Congress,  the  Office  of 
Management  and  Budget,  the  Department  of  Defense,  and  the  military 
services.)  but  efficient  and  effective  implementation  practices  remain 
poorly  understood.  There  is  a  general  belief  that  competition  should  be 
more  widely  utilized,  but  quantitative  support  for  that  belief  is 
lacking. 

In  an  earlier  study1  performed  for  the  Office  of  the  Under 
Secretary  of  Defense  for  Research  and  Engineering  (0U3DRE) ,  Rand 
presented  some  limited  evidence  suggesting  that  competition  during  the 
hardware  development  phase  of  acquisition  was  instrumental  in  reducing 
cost  growth,  but  that  the  incidence  of  such  competition  had  increased 
only  slightly  over  the  past  decade.  Rand  then  undertook  for  OUSDRE  a 
follow-on  study  focusing  on  the  incentives  that  may  affect  the  extent  of 
competition  throughout  the  acquisition  cycle.  This  report  presents  a 
policy- level  overview  of  some  factors  that  may  be  inhibiting  more 
widespread  use  of  competition,  and  suggests  some  changes  in  policy, 
regulations,  and  procedures.  It  also  suggests  additional  quantitative 
research  to  provide  acquisition  managers  with  greator  incentives  to  use 
competition,  and  to  help  them  identify  the  circumstances  under  which 
competition  is  likely  to  pay  off. 

This  report  emphasizes  the  role  of  competition  in  the  acquisition 
of  full  weapon  systems,  major  subsystems  and  components.  The  srasults 
should  be  of  interest  to  defense  acquisition  managers  and  policy-level 
executives. 

Edmund  Sew*  et  al.  Acquisition  Do  Hey  Effectiveness;  Depart - 

2|  Defense  Experience  in  the  1970s.  Report  K-2516-Ui&£,  October 

1979. 
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SUMMARY 


This  study  addresses  the  question  of  how  the  Department  of  Defense 
(DoD)  might  improve  the  effectiveness  of  competition  in  the  acquisition 
of  major  weapon  systems.  The  research  approach  was  to  examine 
incentives,  disincentives,  and  uncertainties  regarding  competition  as 
perceived  by  program  managers  and  other  senior  DoD  acquisition 
officials.  This  approach  is  rooted  in  the  belief  that  competition  can 
be  used  wore- effectively  if  acquisition  managers  have  a  clearer 
understanding  of  which  kinds  of  competition  are  most  effective  and  the 
circumstances  in  which  competition  is  likely  to  be  beneficial. 

As  a  system  moves  through  the  acquisition  cycle,  the  application  of 
competition  changes  in  two  major  ways;  in  the  mix  of  bonefits  sought 
from  competition,  and  in  the  cost  of  supporting  such  competition. 

Before  the  beginning  of  full  scale  development  it  is  common  to  find 
competition  among  several  firms  over  a  combination  of  design  concept, 
expected  system  performance  and  delivery  schedule,  and  estimated  costs. 
The  investments  required  to  achieve  such  competition  are  relatively 
small,  and  the  benefits  are  generally  perceived  to  be  large.  In 
contrast,  true  price  competition  is  rarely  found  during  the  procurement 
phase.  Here  the  manager  is  faced  with  two  major  disincentives:  the 
introduction  of  competition  will  usually  require  added  investment  in  the 
current  year  (with  anticipated  savings  several  years  in  the  future),  and 
the  program  is  likely  to  be  lengthened,  incurring  additional  management 
effort  and  workload.  The  net  effect  of  these  disincentives  varies  from 
program  to  program,  but  they  are  rarely  a  trivial  aspect  of  the 
manager's  decision  on  whether  or  not  to  introduce  competition  during  the 
procurement  phase  of  a  weapon  acquisition  program. 

The  justification  usually  postulated  for  introducing  competition  in 
procurement  is  the  opportunity  to  achieve  a  lower  unit  production  cost. 
Thus  it  is  necessary  to  predict  the  effect  on  price  that  can  be 
anticipated  if  competition  is  introduced.  Unfortunately,  th-  existing 
body  of  analysis  has  focused  on  past  procurement  of  relatively  small 
items  (average  unit  cost  less  than  $10,000),  and  it  does  not  provide  an 
adequate  set  of  management  tools  for  estimating  either  the  dollar 


benefits  or  the  costs  of  competitive  reprocur ement .  Furthermore,  the 
literature  indicates  that  much  of  the  conventional  wisdom  about 
competitive  reprocurement  rests  on  shaky  foundations,  and  that  we  may 
know  less  about  competitive  reprocurement  than  we  thought  we  did. 
However,  some  conclusions  seem  warranted; 

o  Savings  on  electronic  items  repx’ocured  competitively  by  the 
Army  have  been  substantial,  but  we  do  not  yet  know  to  what 
extent  thuse  savings  are  dependent  on  (a)  technological 
innovations  in  the  electronics  industry  confined  to  a 
particular  time  period,  (b)  the  Army's  effective  management  of 
electronics  acquisitions,  (c)  the  relative  ease  of  tranferring 
the  necessary  technology  from  cm,.  electronics  firm  to  another, 
(d)  the  existence  of  a  commercial  market  for  similar  items, 
which  motivates  producers  to  invest  their  own  funds  to  ensure 
that  the  technology  transfer  process  is  successful,  or  any  of 
several  other  explanations. 

o  A  major  decision  in  a  competitive  reprocurement  is  when  in  the 
production  run  tc  introduce  the  second  source.  Information 
currently  available  on  earlier  procurements  yields  little, 
useful  guidance  on  this  point. 

o  While  production  costs  tend  to  decrease  with  increasing 
quantities  produced,  suggesting  that  the  original  producer 
should  be  abl«  to  win  a  competitive  award  hands  down,  we  find 
that  more  often  than  not  the  original  producer  loses  when  an 
item  is  reprocured  competitively,  hut  we  do  not  know  why,  One 
earlier  study  presents  some  evidence  suggesting  that  the  profit 
weights  for  sole-source  contracts,  as  do* turd  in  the 
procurement  regulations,  will  motivate  a  •-•aax.ia'ising 

producer  to  set  up  a  production  process  that  is  inefficient, 
then,  when  this  producer  faces  competition  from  a  second 
source,  he  seems  unable  to  shift  to  a  more  efficient  production 
process . 

o  It  is  not  clear  whether  competitive  reprocurement  pays  off  as  a 
financial  investment  on  systems  as  complex  as  missiles,  because 
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there  is  as  yet  no  evidence  that  internal  rates  of  return  are 
high  enough  to  justify  the  drain  on  front-end  funds. 

o  Data  on  competitive  reprocureaents  are  difficult  to  retrieve; 
and  adequate,  complete,  and  consistent  data  are  almost  non¬ 
existent.  If  data  collection,  storage  and  retrieval  systems 
remain  as  they  are,  our  quantitative  understanding  of  the 
competitive  reprocurement  process  is  likely  to  remain  weak  into 
the  1990s. 

We  thus  conclude  that  existing  research  provides  neither 
quantitative  nor  qualitative  guidance  for  designing  price-competitive 
reprocurement  strategies*-one  of  the  simplest,  and  certainly  the  most 
quantifiable,  uses  of  competition.  To  correct  this  situation,  further 
work  is  needed  on  methods  for  forecasting  and  evaluating  the  costs  and 
benefits  of  various  competitive  strategies-- not  just  the  effect  on  price 
of  competition  in  the  reprocurement  phase.  Lacking  such  methodologies, 
program  personnel  are  unable  effectively  to  justify  competition 
expenditures  and  the  CoD  is  unable  to  develop  criteria  for  selective 
application  of  competition.  Further,  without  knowing  how  to  evaluate 
competition  in  the  context  of  acquisition,  decisionmakers  will  tend  to 
focus  on  the  degree  to  which  competition  is  or  is  not  applied,  instead 
of  how  much  it  has  or  has  not  increased  the  effectiveness  of  systems 
acquisition.  The  development  of  such  methodologies  will  be  neither 
simple  no;  rapid,  given  the  difficulties  of  retrieving  adequate 
inform  tint  about  -peat  programs. 

A  simple  bus  valuable  first  step  would  be  to  ensure  that  better 
records  are  retained  on  current  competitive  acquisition  actions,  so  that 
future  e*»a  lysis  can  benefit  from  an  adequate  data  base,  Such  a  data 
base  shoo  <o  enable  analysts  to  focus  on  a  broad  set  of  characteristics 
of  individual  weapon  systems  as  determinants  of  contract  outcomes,  as 
oppo*  *d  to  looking  only  at  learning  curves  and  cost  relationships,  The 
degree  of  tec*.nieai  sophistication  and  the  advance  it  represents  over 
the  previous “generation  system  are  potent ialy  important  attributes.  The 
role  of  the  contract  itself  should  also  be  considered  in  more  detail. 
Even  when  contracts  are  classified  ss  being  of  ;he  same  type,  many 


aspects  may  vary  widely,  such  as  escalation  clauses,  penalties,  award 
fees,  payment  ceilings,  and  so  forth. 

Finally,  the  underlying  theory  of  how  competion  should  function  in 
weapon  system  acquisition  seems  to  be  inadequately  developed.  Research 
on  the  development  and  refinement  of  such  a  theory  would  be  extremely 
valuable  if  it  could  provide  a  solid  framework  to  guide  the  collection 
of  data  and  the  design  of  future  data  analysis. 
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I ,  INTRODUCTION 

In  its  1972  report.,  the  Congressional  Commission  cn  Government 
Prop.urement  made  recommendations  ranging  from  the  need  for  a 
"competitive"  rather  than  a  "maximum"  number  of  sources  at  the  proposal 
stage,  to  the  role  of  competition  in  the  later  stages  of  system 
acquisition.  The  Commission  dealt  at  length  with  what  it  perceived  to 
he  the  need  for  introducing  competition  •before  the  buyer  bad  settled  on 
a  detailed  technical  approach  to  meeting  an  established  requirement. 1 

After  receiving  the  Commission's  report  in  1972,  the  Congress 
created  within  the  Office  of  Kanagemeut  and  Budget  an  Office  of  Federal 
Procurement  Policy  and  empowered  it  to  establish  broad  policy  for 
governmental  acquisition  and  procurement  action.  OMB  Circular  A*109, 
promulgated  in  1976,  encouraged  design  competition  throughout  the 
acquisition  process  and  urged  that  nan-competitive  practice  he  the 
exception  rather  than  the  rule,  especially  in  the  early  phases  of 
acquisition  before  major  resource  commitments  were  made,  The  Circular 
also  mandated  that  justification  had  to  be  provided  to  the  Congress  if 
competition  was  net  used  prior  to  full-seals  development.  Moreover, 
competition  was  explicitly  allowed  through  development  and  procurement , 
with  the  objective  in  the  later  phases  of  acquisition  apparently  being 
focused  m  price  reduction  rather  than  the  design  excellence  sought 
earlier. 

In  parallel  with  the  broad  policy  initiatives  of  the  Congress  and 
£&1.  policy  analysts  devoted  considerable  attention  to  the  question  of 
hew  competition  could  best  fee  applied  to  the  special  prefelons  of  the 
department  of  oeiesss.  Asy  -sewaoetsfele  scholarly  study  or  ©overisieot* 
sponsored  review  of  weapon  cysts®  acquisition  policies  and  practices 
during  the  19?bs  w-as  certain  to  include  a  section  on  competition, * 

*  Report  of  the  Commission  on  teveranens  frecurement *  IM, 
SevemHent  Printing  iff f see  ho.  December  1992;  see  especially 

Vel .  2  CPart  S* -Acquis it isft  of  fcesaarch  and  Development^  Part  £-* 
Acquisition  of  Major  Systems!, 

4  See,  for  example.  J.  Ronald  Fox-,.  Arming  America  fCssferidpi. 
harvard  University  Press,  197*1*  pp  Flectronics*y:  A  Study  of 

Military  Electronics  with  Particular  Reference  to  dost  and  Reliability 
f  SOA  Report  Ss‘l9F,  January  197*!,  pp.  tyi-212;  and  the  Defense  Science 
heard  1979  Sumter  Study.  Final  Report. 


Such  studies  generally  have  concluded  that  competition  is  a  good  thing 
and  more  competition  would  be  better  yet. 

For  more  than  20  years,  the  DoD  has  attempted  in  various  ways  to 
increase  the  role  of  competition  in  the  weapons  acquisition  process. 
Secretary  of  Defense  Robert  S.  McNamara  viewed  procurement  competition 
as  a  key  element  of  the  management  innovations  he  introduced  during  the 
early  1960s.  After  Deputy  Secretary  of  Defense  David  Packard  became 
responsible  for  defense  acquisition  policy  in  1969,  he  issued  a  series 
of  memoranda  that  not  only  supported  competition  among  developers 
involved  in  the  creation  of  new  hardware  but  extended  the  emphasis  to 
include  subsystems  as  well  as  full  weapon  systems.  He  instructed  the 
three  military  services  to  adopt  these  memoranda  as  official  policy  and 
to  modify  Service  policies  accordingly.  The  eventual  DoD  Directive,3 
however,  muted  Packard's  initial  endorsement  of  competition. 

The  various  0MB  and  DoD  acquisition  policy  directives  issued  since 
then  have  generally  emphasized  competition  during  the  earlier  phases  of 
development.  0MB  Circular  A-1091*  advocates  "competitive  exploration 
of  alternative  system  design  concepts."  However,  competition  later  in 
the  acquisition  cycle  is  rarely  mentioned,  and  then  only  advocated 
"whenever  economically  beneficial."  No  specific  mention  is  made  of 
competition  during  the  production  phase,  nor  is  there  any  hint  that  such 
competition  may  be  justified  for  reasons  other  than  cost  savings.  DoD 
Directive  5000.1  and  DoD  Instruction  5000.2  repeat  the  same  theme  in 
slightly  different  words. 

The  military  services  are  in  at  least  formal  compliance  with  0MB 
and  OSD  policy,  and  in  some  cases  make  an  even  stronger  endorsement  of 
competition.  Army  Regulation  1Q00-15  emphasized  competition  oven 
before  Circular  A-109  became  official  executive  branch  pc. icy.  The  1975 

i  DOD  Directive  5000,1,  Major  System  Acquisition,  1971  (revised 
most  recently  in  March  1980). 

*  Circular  A-109,  Major  System  Acquisitions .  Office  of  Management 
and  Budget,  April  5,  1976. 

s  Regulation  AR  1000-1,  Basic  Policies  for  Systems  Acquisition. 
Headquarters,  Department  of  the  Army,  April  1,  1978. 
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version  of  AR-1000-1  stated  in  part  that  the  early  stages  of  the 
development  process.  .  ."should  include  fabrication  and  testing  of 
competitive  prototypes.  .  ."  (Section  5a)  and  that  programs  should 
"assure  contractor  competition.  .  .  [early]  in  full-scale  production 
.  .  .  based  on  a  technical  data  package"  (Section  5d).  The  Navy  and  Air 
Force  made  no  independent  directives  with  respect  to  competition,  but 
attached  DoD  Directive  5000.1  to  their  own  regulations,  and  endorsed  its 
applicability. 

With  such  an  extensive  body  of  policy  directives  issued  over  the 
past  decade,  could  full  implementation  have  been  far  behind?  For  the 
Department  of  Defense  that  question  is  surprisingly  hard  to  answer > 
partly  because  the  extent  to  which  competition  has  actually  been  used  in 
weapon  system  acquisition  is  difficult  to  determine,6  and  partly 
because  of  uncertainty  about  the  circumstances  in  which  competition  is 
reasonable  and  desirable.  Competition  is  actually  mandated  only  for 
that  part  of  the  acquisition  cycle  that  precedes  the  start  of  full  scale 
development  (FSD).  er  the  start  of  FSD,  competition  becomes 
discretionary.  NonetneTess,  while  the  actual  and  the  preferred  extent 
of  competition  remain  unclear,  it  is  generally  believed  inside  and 
outside  the  Department  of  Defense  that  more  competition  is  needed  and 
would  be  economically  benefit '  al . 

THE  STRUCTURE  OF  COMPETITION 

The  quantitative  evidence  about  the  influence  of  competition  on 
acquisition  is  subject  to  theoretical  and  empirical  qualification,  yet 
most  observers  argue  that  competition  produces  many  significant 
benefits; 

o  Improved  product  quality. 

o  Lower  unit  costs. 

o  Faster  rates  of  learning  by  the  manufacturer, 

*  A  description  of  the  measurement  difficulties  is  contained  in 
White  and  Meyers,  Cuajietition  in  Dp!)  Acquisition,  Logistics  Management 
Institute,  Washington,  D.tf. ,  May  *19?? . 
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o  Greater  technological  progress, 
o  Enhanced  industrial  productivity, 

o  Enlarged  surge  and  mobilization  capacity, 

o  More  equitable  process  of  awarding  acquisition  contracts. 

In  light  of  such  claims,  it  is  important  to  consider  what  exactly 
is  meant  by  "competition.*'  In  the  context  of  systems  acquisition.  Hall 
and  Johnson,7  for  example,  chose  to  distinguish  between  pure  price 
competition  and  competition  which  included  non-price  factors  such  as 
technical  design.  The  former  they  called  competition  and  the  latter 
rivalry,  from  an  acquisitions  viewpoint,  that  distinction  represents  a 
clear  categorization  of  factors  on  which  to  base  competition,  but  it 
still  suffers  several  problems.  First,  the  distinction  has  not  been 
widely  accepted  in  the  acquisitions  community  or  in  its  literature,  and 
second,  very  few  cases  arise  which  are  not  rivalvcus.  In  most  cases, 
factors  such  as  system  performance  and  delivery  schedule  play  a  key  role 
in  the  competition  for  the  contract.  Thus,  we  consider  the  Hall  and 
Johnson  definition  of  competition  too  limited  for  use  here. 

In  the  economic  literature,  competition  and  rivalry  also  are  key 
concepts,  but  in  a  somewhat  different  way  than  .in  the  Hall  and  Johnson 
article.  Competitive  firms  art  those  which  have  a  very  restricted 
ability  to  affect  price  or  product  characteristics  or  to  undertake  any 
strategic  behavior.  Riva Irons  firms  are  those  which  undertake  pricing 
strategies,  new  advertising,  product  development,  and  other  actions 
which  may  greatly  enhance  their  market  positions.  From  this  viewpoint, 
virtually  all  weapon  system  acquisitions  would  be  considered  as 
rivalreus  rather  than  competitive.  Again,  however,  that  concept  also  is 
not  used  in  the  acquisition  community  and  would  lead  to  confusion  if 
used  here. 

Furthermore,  in  the  government's  acquisition  of  major  systems, 
subsystems,  and  technologically  sophisticated  components,  the  market 
differs  from  the  economic  structural  ideal  in  several  essential  ways: 

R.  Hall  and  K .  E.  Johnson.  A  Review  of  Air  Force  Procure¬ 
ment.  1965-1964.  The  Rand  Corporation!  RM-4S00-PR,  ?iay  ~1965. 


0)  the  filial  products  do  not  exist  at  the  time  developers  are  selected, 
and  they  usually  do  not  exist  in  final  form  when  producers  are  selected; 
(2)  there  are  very  few  buyers  for  these  products  (although  it  is  an 
error  to  view  the  "marketplace"  as  always  having  only  one  buyer);  (3) 
the  buyers  have  very  imperfect  information  concerning  the  prices  and 
functional  specifications  of  the  product,  their  own  need  for  the  product 
vi.e.,  the  threat  is  uncertain  and  changing),  and  the  relevant  budgetary 
cciistraincs,  especially  in  future  years;  and  finally,  (4)  the  entry  and 
exit  of  firms  in  this  market  is  often  slow  and  costly. 

To  avoid  confusion  among  the  several  concepts  above,  we  will  use 
i-he  term  competition  in  its  broad,  generic  sense.  Any  acquisition 
arrangement  that  requires  more  than  one  firm  to  compete  for  a  particular 
contract  will  M  called  competitive.  When  we  consider  competition  in 
development  or  production,  in  price  or  non-price  factors,  or  in  other 
cases,  it  will  be  noted.  Further,  contracts  that  do  not  allow  direct 
competition  will  not  be  considered  competitive  (for  example,  sole-source 
follow-ons  to  competitive  contracts,  awards  in  response  to  unsolicited 
proposals,  multiple  sole-source  contracts,  and  so  forth),' 

A  careful  reading  of  Congressional  testimony  loads  one  to  suspect 
that  in  the  legislative  context  tba  term  "competition"  usually  implies 
"price  competition."  But  the  usq  of  price  competition  is  uncommon  (at 
least  since  tha  total  package  proenrement  approach  fell  into  disfavor). 
True  price  competition,  unaffected  by  non-p-iee  elements,  does  not 
generally  coma  into  its  own  until  the  reprocuretaent  phase* “after  at 
least  one  full-scale  production  run,  and  even  then  factors  other  than 
price  are  usually  included. 

When  discussing  competition  or*  recording  procurement  actions  as 
competitive,  the  Bod  is  typically  referring  to  forms  of  non-price 
competition.  Daring  source  selection  for  a  development  program, 
competing  proposals  are  evaluated  for  technical  and  design  features. 
Price  proposals  also  fray  be  weighed,  but  the  final  price  is  almost 
6lwayo  negotiated  after  the  source  selection. 

1  these  may  have  outcomes  as  if  they  were  competitive  contracts 
and  ‘bus  may  be  propely  classified  as  such.  For  analytic  simplicity,  we 
remain  with  the  above  definition. 


Figure  1  depicts  the  several  types  of  competition  that  may  be 
employed  when  acquiring  weapon  systems.  Along  the  bottom  of  the  figure 
we  indicate  the  sequence  of  stages  by  which  a  major  system  is  created 
and  deployed--from  concept  formulation  through  development  (prototype 
and  full-scale)  to  production  and  procurement.  The  ways  in  which 
competition  (over  design  or  price  or  both)  may  be  encouraged  can  be  seen 
for  each  vertical  segment  representing  a  stage  of  the  acquisition 
process . 

During  concept  formulation,  competition  may  concern  only  the 
design,  as  is  often  the  case  in  the  kind  of  R&D  activities  funded  by  the 
Defense  Advanced  Research  Projects  Agency,  or  (more  commonly)  both 
design  and  price.  At  this  point  treating  competition  as  "principally" 
concerning  either  design  or  price  can  be  quite  misleading. 

During  the  demonstration  and  validation  phase,  competition 
generally  focuses  on  both  design  and  price.  The  design  specification 
typically  is  not  yet  fixed,  and  opportunities  still  exist  to  trade 
performance  for  price  (such  tradeoffs  may,  of  course,  take  place  in  the 
context  of  what  the  DoD  calls  a  "design-to-cost"  goal).  Competition 
among  contractors  may  still  bo  achieved  by  the  buyer  through  the  use  of 
competitive  hardware  demonstration  programs.  The  Air  Force's 
Lightweight  Fighter  program  (prior  to  selection  of  the  F-16A  for  full 
scale  development)  if  an  example. 

During  the  full-scale  development  phase,  competition  may  bo 
retained  through  the  use  of  a  second  source  (as  its  the  case  of  the  Army 
UH-60  helicopter).  Otherwise,  solo-source  development  may  bo  completed 
and  a  second  source  qualified  after  an  initial  "learning  buy."  However, 
such  re introduction  of  price  competition  may  be  costly  for  the  buyer  if 
he  has  not  anticipated  it  in  his  original  acquisition  strategy. 

‘'Breakouts"*  for  second-sourcing  of  subsystems  and  components 
during  production  also  have  occurred,  sometimes  to  achieve  better  price 
or  quality  and  sometimes  to  ensure  an  adequate  industrial  base. 

Although  the  data  are  quite  imperfect,  the  dollars  committed  by  the 

""*"*"*  In  this  acquisition  strategy,  the  competitive  acquisition  of  a 
subsystem  or  component  is  managed  directly  by  the  Service  Project  Office, 
rather  than  by  the  prime  system  contractor. 


Fig.  1— Types  of  competir. 
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Federal  Government  for  such  competitive  breakouts  probably  have  not 
amounted  to  more  than  10  or  15  percent  of  total  procurement  funds  for 
major  systems.  Of  course,  the  p?:Jme  contractor  on  a  major  weapon  system 
may  obtain  competitive  bids  from  its  svbr  jil'  rectors  and  suppliers  at  the 
outset  of  a  development  program.  Depending  oji  the  t-ype  of  hardware 
involved,  the  prime  contractor  m  ,T  -..ontinue  to  use  price  competition  for 
some  portion  of  his  annual  purchase-..  1’h-'  tenoi.u  to  Government  of 
maintaining  that  competitive  environment  vai>  with  contract  type  and 
with  the  susceptibility  of  the  contract  to  modifier.  ;.c:i  in  the  course  of 
the  program. 

For  most  acquisition  programs,  existing  .  rr>teu-.  ec for  the 
same  mission  as  that  planned  for  the  new  weapon  systc.  T:.u^,  at 
various  stages  during  the  production  or  reprocurement  p  -r.^ss, 
competitive  options  exist  which  do  not  involve  a  single  acquisition 
program,  but  instead  involve  product  improvement  alternatives  based  upon 
existing  equipment.  Examples  include  the  various  versions  of  the  F-4 
and  B-52  aircraft  which  survived  the  competition  with  other  aircraft 
options.  Competition  of  this  sort,  however,  transcends  the  charter  of 
individual  program  managers.  Once  hardware  development  has  begun,  such 
competition  can  be  vigorous  only  at  the  level  of  the  Service  heads,  the 
Dot*,,  and  the  Congress. 

The  possibility  of  introducing  affective  price  competition  during 
the  reprocurement  phase  is  dependent  on  appropriate  preparations  having 
been  made  early  in  the  program,  whilo  a  competitive  environment,  still 
exists,  (The  purchase  of  a  Technical  Data  Package  (TDP)  will  suffice  in 
some  instances',  arrangements  for  technical  assistance  to  and 
qualification  of  additional  producers  will  be  needed  in  others.)  For 
certain  electronic  items,  it  appears  that  the  purchase  of  a  TDP  during 
tho  development  phase  and  an  Invitation-For-Bid  (1FB)  competition  aftor 
the  first  few  sole-sourco  production  runs  generated  substantial  unit- 
price  reductions  in  the  l%0s.la  For  such  othor  items  as  missilo 
guidance  and  control  units,  the  noccssary  and  sufficient  conditions  for 
\e  effective  use  of  pvico  competition  in  the  roprocuremont  phase  are 
less  clear.  For  systems  as  complex  as  aircraft,  thoso  conditions  are 
themselvQs  complex  and  must  bo  addressed  very  early  in  tho  acquisition 
cycle,  Fusion- fission,  loader -follower,  and  diructed  licensing  are 

Vb ~Soe "Sec  !“lil  for  details. 
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three  acquisition  strategies  which,  if  implemented  appropriately, 
provide  options  for  introducing  competition  in  the  production  phase  of 
major  systems.11  These  are  complex  strategies,  however,  and  we  have 
little  experience  in  their  application  to  major  weapon  systems. 

This  overview  of  some  of  the  major  issues  related  to  competition  is 
included  to  provide  a  framework  for  the  subsequent  discussion  of 
incentives  and  disincentives  for  effectively  using  competition.  It 
suggests  the  following  conclusions: 

o  Formal  policy  directives  support  the  limited  use  of 

competition,  although  the  exact  types  of  competition,  the  focus 
of  application,  and  the  benefits  expected  are  ambiguous, 
o  Competition  may  be  employed  to  achieve  benefits  other  than,  or 
in  addition  to,  price  reductions, 
o  Numerous  opportunities  occur  throughout  the  acquisition  cycle 
for  the  judicious  introduction  and  continuation  of  competition, 
o  It  is  difficult  to  measure  the  extent  to  which  competition  has 
actually  been  employed  in  weapon  acquisition,  but  many  believe 
that  competition  should  bo  used  more  extensively. 


STUDY  OBJECTIVES  AND  APPROACH 

The  objective  of  this  report  is  to  suggest  how  the  Dob  might 
stimulate  more  effective  competition  in  major  acquisition  programs. 
Impressed  by  the  difficulty  experienced  by  previous  researchers  in 
measuring  the  extant  or  the  benefits  of  competition  in  weapon 
acquisition,  we  adopted  a  different  approach:  i.o.,  we  attempt  to 
identify  tho  factors  leading  to  the  use  or  rejection  of  competition,  in 

11  Leader -follower  and  directed  licensing  are  variations  on  a 
plan  whore  one  company  develop*  and  than  provides  the  technical 

assistance  necessary  for  a  second  company  to  produce  it,  thus  establish¬ 
ing  a  competitive  posture.  In  one  version  the  developer  (leader)  is 
paid  a  foe  for  tho  technology  transfer  service,  aud  in  the  other  version 
the  developer  collects  a  royalty  on  seeor.d*aeurce  production.  Fusion- 
fission,  a  relatively  new  technique,  requires  two  firms  to  share  the 
development  task  and  then  each  sots  up  a  full  production  capability  and 
competes  for  production  lots. 
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the  belief  that  such  information  would  yield  insights  on  how  to  enhance 
the  use  of  effective  competition.  What  is  sought  here  is  a  management 
perspective  on  competition,  focusing  on  management  as  a  decision  process 
that  selects  from  a  variety  of  acquisition  strategies,  sometimes 
choosing  competition,  sometimes  not.  Issues  to  be  addressed  include  how 
managers  view  competition,  what  problems  arise,  how  serious  they  are, 
what  benefits  can  be  ixpected,  and  how  the  effectiveness  of  competition 
can  be  enhanced. 

Wo  undertook  a  series  of  interviews*  with  key  management  personnel 
to  learn  how  they  currently  perceive  the  benefits  and  costs  of  employing 
competition  in  the  various  phases  of  acquisition.  Programs  in  the 
sample  included  aircraft  (F-16,  F-18),  a  helicopter,  (AH-64)  and  other 
systems  (Advanced  Self-Protoctiun  Jammer  (ASPJ),  M-196  Howitzer).  In 
addition,  extensive  review  of  background  work  on  competition  bolstered 
the  empirical  research.  The  results,  however,  must  be  taken  as 
illustrative  and  tentative.  No  validation  of  the  interviews  was 
undertaken,  and  the  sample  of  personnel  and  programs  was  small.  Still, 
the  results  reflect  the  concerns  of  an  important  set  of  acquisition 
participants  and  provide  insights  and  suggestions  for  improving  the 
effectiveness  of  competition.  These  management  perspectives  are 
described  in  Sec.  11, 

We  had  planned,  at  the  beginning  of  our  research,  to  include  a 
brief  review-  of  existing  quantitative  research  on  the  savings  achieved 
through  competitive  reprocurement.  The  topic  has  received  considerable 
attention  by  others  during  the  past  decade,  so  we  did  not  expect  that  it 
would  be  necessary  for  us  to  devote  much  effort  to  it,  One  consequence 
of  our  interviews  and  discussions.  However,  was  the  realisation  that  key 
decisionmakers  place  little  confidence  in  either  the  precision  or 
relevance  of  such  estimates  of  savings,  We  found  grounds  for  this 
skepticism  in  the  uneven  quality  of  the  quantitative  work  that  has  been 
done,  Analysts  working  on  the  topic  do  not  agree  on  the  extent  of 
savings  realised  on  completed  competitive  r ^procurements ,  or  even  on  how 
such  savings  should  be.  estimated.  And  when  it  comes  to  forecasting 
savings  on  contemplated  competitive  rep rocur aments,  analysts  do  not  even 
agree  among  themselves  on  whether  it  can  be  done,  let  alone  on  how  to  do  it. 
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This  uncertainty  seemed  to  be  a  critical  element  in  any  attempt  to 
foster  the  use  of  effective  competition  in  defense  acquisitions.  If 
realized  savings  (or  losses)  on  past  competitive  reprocuremonts  cannot 
be  measured  with  much  precision,  and  the  expected  benefits  (to  be 
achieved  several  years  hence)  on  prospective  competitive  reprocurements 
cannot  be  estimated  with  any  confidence,  then  the  known,  near-term  costs 
and  risks  of  introducing  competition  may  well  appear  prohibitive,  and  in 
some  cases  rightly  so.  We  therefore  undertook  an  extensive  and  critical 
review  of  the  quantitative  literature  on  competitive  reprocurcmont ;  its 
implications  for  the  use  of  competition  are  described  in  Sec.  IIT. 

Finally,  in  Sec.  IV  we  summarize  our  results,  state  our  findings, 
and  offer  recommendations  on  how  to  improve  the  use  of  competition  in 
the  acquisition  of  major  weapon  systems. 
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II.  MANAGEMENT  PERSPECTIVES 


Acquisition  policy-makers  at  the  highest  levels  of  the  DoD  and  the 
Services  strongly  espouse  the  increased  use  of  competition  in  system 
acquisition.  However,  whether  competition  is  likely  to  be  implemented 
to  a  greater  extent  depends,  in  part,  upon  hr>v  closely  the  anticipated 
benefits  correlate  with  the  fundamental  objectives  and  perceptions  of 
the  acquisition  managers.  For  example,  <ioos  competition  help  a  system 
to  be  deployed  faster,  minimize  technical  risks,  develop  a  wider 
political  base,  help  acquisition  personnel  get  promoted,  ease  financial 
and  administrative  problems,  or  speed  the  advance  of  technology?  Do 
program  participants  recognize  and  have  confidence  in  the  benefits  of 
competition,  based  on  sound  evidence? 

It  is  far  from  obvious  that  the  answers  to  such  questions 
invariably  favor  the  use  of  competition.  Thus,  if  we  want  to  increase 
the  effactivenesa  of  competition  it  may  be  necessary  to  improve  our 
understanding  of  how  managers  and  other  acquisition  executives  perceive 
the  effects  of  and  what  actions  might  bo  taken  to  shift 

those  percept <v.  a  direction  favoring  greater  use  of  competition, 
where  appropriate. 


«&TKQBQL00¥ 

To  gain  a  better  understanding  of  what  program  management  personnel 
perceive  to  be  the  advantages  and  disadvantages  of  competition,  we 
conducted  a  series  of  interviews  with  the  Service  people  involved  in 
designing  and  carrying  out  a  program  acquisition  strategy  in  their 
Program  Offices.  The  personnel  in  each  case  were  either  program 
managers  or  deputy  program  managers,  along  with  key  Program  Office 
personnel.  The  programs,  listed  *n  Table  1,  span  all  Services  and 
represent  a  variety  of  system  types.  Interviewees  were  chosen  on  the 
basis  of  the  variety  of  competition  experiences  they  represent. 

Questions  were  designed  to  assess  the  extent  to  which  competition 
was  used  on  the  program,  how  it  was  employed,  what  the  circumstances 
were,  what  factors  and  personnel  were  involved  in  the  decision,  and  what 
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Table  1 

PROGRAMS  DISCUSSED  IN  THE  INTERVIEWS 

AH-64  Helicopter 
LWF/F-16  Aircraft 
HARPOON  Missile 
MAVERICK  Missile 
SPARROW  Missile 
F-18  Aircraft 

Advanced  Self  Protection  Jammer  (ASPJ) 

M-19S  Howitzer 

]  the  consequences  of  its  use  were.  The  questions  did  not  seek 

j!  quantitative  information,  but  rather  insights  into  the  difficulties  and 

f  advantages  of  employing  competition.  Further,  the  emphasis  was  on  full 

systems  (although  subsystems  and  components  were  discussed)  in  all 
phases  of  the  acquisition  cycle. 

Vhile  the  sample  is  small,  the  people  interviewed  were  very 
familiar  with  their  own  programs,  and  frequently  had  a  great  deal  of 
experience  with  weapon  system  acquisition.  Thus,  their  descriptions  are 
considered  representative  of  widely  held  perceptions  regarding  the  use 
of  competition  in  acquisition  p^ogr was. 

The  interviews  explored  four  main  areas i 

q  Experience  with  competition '-description  of  how  and  why 
competition  was  used,  and  to  what  effect, 
o  Incentives  and  disincentives --percept ions  of  how  competition 
supports  or  threatens  the  objectives  sutd  needs  of  individuals 
in  acquisition  management. 

o  Current  climate --percept ions  of  who  key  figures  are  in 
,  setting  acquisition  policy  and  how  effective  certain  policies 

have  been. 


o  Background  and  context--perceptions  on  objectives  of  the 

Program  Office  and  its  position  in  the  acquisition  hierarchy, 
what  support  was  available,  what  contracting  techniques  were 
used,  and  what  lessons  were  considered  relevant  for  future 
program  managers. 

In  addition  to  the  Program  Office  personnel,  we  interviewed  persons 
involved  in  Federal  competition  policy  (OMB,  Congress)  in  order  to  gain 
a  higher  level,  policy-oriented  perspective. 

Several  caveats  should  be  noted.  First,  the  authors  have  made  no 
effort  to  verify  all  of  the  observations.  For  the  purpose  of  examining 
the  perceptions  and  motivations  of  acquisition  participants,  such 
verification  was  unnecessary.  A  mistaken  belief  in  the  disadvantages  of 
competition  can  be  a  very  effective  barrier  to  its  use. 

Another  caveat  is  that  the  views  reported  here  were  expressed  by  at 
least  one  of  the  top-level  Program  Office  personnel  interviewed,  but  not 
necessarily  by  all.  It  is  unlikely  that  all  would  agree  with  all  of  the 
views,  but  mast  were  expressed  more  than  once.  Further,  sine©  most 
decisions  involved  a  balancing  of  perceived  advantages  and 
disadvantages,  instances  were  found  where  similar  applications  of 
competition  were  viewed  in  conflicting  terms  by  different  managers. 

Thus,  the  objective  of  the  work  was  to  develop  a  cumulative  impression 
of  the  managers*  problems,  recognising  that  internal  inconsistencies  are 
bound  to  occur. 

The  following  discussion  of  the  perceived  disadvantages  of 
competition  may  appear  to  draw  a  dismal  picture  of  its  effects.  This 
results  from  a  focus  on  the  negative  effects,  prompted  by  our  interest 
in  the  perceived  disincentives  for  competition.  Host  of  those 
interviewed  fevered  the  use  of  competition,  albeit  with  discretion  and 
careful  planning.  Thus,  the  following  di**ussion  should  not  be 
interpreted  as  a  critique  of  competition,  but  rather  as  an  an  overview 
of  the  perceived  barriers  to  better  use  of  competitive. 
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RESULTS 

The  barriers  to  eoopetitioa,  as  perceived  by  senior  program 
acquisition  personnel,  may  be  summarised  under  three  headings; 

1.  Additional  time  and  money  needed. 

2.  Extra  management  complexity  and  effort  required. 

3.  few  near-term  benefits  and  incentives  expected  with  high 
confidence. 


Each  of  these  barriers,  or  disincentives,  is  discussed  separately  below. 
Throughout  the  discussion  an  attempt  will  be  made  to  distinguish  among 
different  problems  that  arise  during  different  phases  of  the  acquisition 
cycle. 


Time  and  Honey 

At  almost  every  phase  in  the  acquisition  cycle,  and  for  almost 
nvery  kind  of  competition,  additional  current-year  investment  is 
required;  over  and  above  what  would  be  necessary  far-  a  sole-source  award. 
Boring  the  planning  phase  such  funds  are  relatively  small  in  absolute 
terms,  although  large  in  comparison  to  the  overall  funds  available  in 
that  feyd.pt  category.  More  important  is  the  fact  that  competition 
during  the  concept  formulation  and  analysis  phase  is  a  well-established 
tradition,  so  that  funding  for  multiple  sources  is  relatively  easy  for  a 
manager  to  obtain, 

When  the  programs  moves  to  the  full-scale  hardware  development 
phase,  the  magnitude  of  the  funding  required  fepf  a  second,  competitive 
source  bscoses  large  in  both  relative  and  absolute  teres.  Furthermore, 
while  general  statements  supporting  competition  occur  at  every  level  in 
the  defense  establishseot,  this  does  not  mean  that  everyone  concerned 
with  a  particular  program  will  he  willing  to  put.  up  the  money  for 
competition.  When  the  funding  required  to  support  a  second,  competitive 
source  reaches  the  level  of  tens  or  hundreds  of  nil  liens  of  dollars, 
etthofiaatioo  will  have  to  cose  free  higher  up  the-  chain  of  coonand, 
this  means  that  many  people  will  have  to  few  ‘"sold"  on  the  cospet stive 
action,-  At  every  level  in  the  organisation  there  will  he  Sene  who  are 


sympathetic  to  the  request  for  funds,  but  others  will  see  themselves  as 
competing  for  the  same  funds.  Some  groups  will  tend  to  underestimate 
the  difficulty  of  developing  a  particular  system  or  have  an  interest  in 
fielding  it  very  quickly,  and  thus  will  resist  competition  during  full- 
scale  development  as  a  waste  of  time  and  money.  The  situation  is  even 
more  complex  in  multi-service  programs  where  all  the  Services  must  agree 
to  put  up  the  extra  money. 

When  substantial  amounts  of  money  are  involved,  the  DoD  and  the 
Congress  must  be  sold  on  the  competition  as  well.  When  there  is  no 
great  pressure  for  competition  and  when  other  acquisition  initiatives 
are  being  emphasized,  those  agencies  can  be  difficult  to  convince. 
Congress  tends  to  dislike  programs  with  heavy  front-end  cost,  and  other 
less  obvious  political  problems  sometimes  intrude.1  Also,  funding 
requests  are  reviewed  by  four  different  Congressional  committees  which 
do  not  automatically  coordinate  their  decisions,  so  each  must  be 
persuaded  separately.  It  is  not  unusual  for  one  committee  to  support  a 
competition  and  another  to  delete  the  funds  for  it. 

Further,  once  funding  for  a  competition  is  approved,  there  is  no 
guarantee  that  the  funding  will  be  maintained.  Money  for  competitive 
development  programs  is  a  prime  target  in  a  budget  squeeze.  Even  if 
high-level  support  for  competition  is  gained  when  the  funding  is  first 
made  available,  there  is  no  guarantee  that  the  support  will  continue. 

In  the  Services  and  in  the  DoD  there  are  frequent  changes  in  top-level 
personnel,  and  when  new  people  take  over  they  often  change  priorities, 
Written  policv  supporting  competition  remains  fairly  consistent,  but 
interest  in  competition  changes  with  personnel.  The  result  is  that  it 
can  be  difficult  to  maintain  all  the  funding  necessary  to  conduct  a 
competitive  development  program. 

Competition  can  slow  the  program  because  of  the  time  involved  in 
testing  and  source  selection  or  in  qualifying  a  second  contractor. 
Schedules  can  also  lengthen  because  of  the  increased  program  complexity 

1  See  Michael  D.  Rich,  Competition  in  the  Acquisition  of  Major 
Weapon  Systems :  Legislative  Pursued. ives ,  R-205S-PK,  The  Rand  Corpora¬ 
tion,  Novomber  1976 
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and  increased  bureaucratic  involvement*  caused  by  competition. 

Because  program  costs  tend  to  increase  faster  than  general 
inflation,  increased  program  length  risks  increasing  cost.  The  risk  of 
increased  program  length  is  itself  a  disincentive  to  competition  because 
there  is  usually  a  strong  desire  to  deploy  the  system  as  rapidly  as 
possible. 

During  the  production  phase  the  funding  required  to  qualify  a 
second,  competitive  source  appears  to  pose  less  of  a  problem,  at  least 
for  less  complex  systems  or  components.  Perhaps  this  is  because  by  the 
time  the  progx-ara  is  in  production  all  major  conceptual  issues  have  long 
since  been  resolved,  and  attention  is  more  easily  focused  on  the  task  of 
efficiently  producing  the  system.  Furthermore,  there  is  some  belief 
that  clear  evidence  of  financial  benefit  exists  for  competitive 
reprocureraent  (but  see  Sec.  Ill  of  this  report). 

Extra  Management  Effort 

A  commonly  mentioned  problem  with  competition  is  that  it  increases 
the  workload  of  the  Project  Office.  This  extra  work  stems  from  two 
sources.  First,  if  a  competition  is  to  be  beneficial,  considerable 
planning  for  the  competitive  steps  is  necessary.  The  request  for 
proposal  (RFP)  must  be  prepared  and  the  source  selection  process  must  be 
designed.  The  Program  Office  must  comply  with  certain  regulations 
designed  to  ensure  the  fairness  of  the  competition.  This  involves 
special  security  to  deal  with  "competition  sensitive"  material,  special 
reports,  etc.  It  also  introduces  the  possibility  of  lawsuits,  disputes, 

Conventional  management  practices  followed  by  most  Program  Qf- 
fines  do  not  seem  to  exploit  the  fotei^tial  advantages  of  design  or  price 
competition.  Sven  where  true  competition  exists,  and  the  competing 
firms  have  adequate  incentives  to  perform  at  their  highest  possible  lev¬ 
el  of  capability,  the  Program  Offices  tend  to  "manage"  each  contractor 
as  if  it  ware  a  soJ«  source.  The  occasional  exception  to  this  rule 
f.e.g. ,  the  prototype  phase  of  the  Lightweight  Fighter  Program)  have 
demonstrated  that  dramatically  austere  management  practices  can  be  suc¬ 
cessfully  used  by  the  Service  Program  Office  under  ^ome  circumstances. 
Adoption  and  general  acceptance,  of  management  practices  more  suitable 
for  dual-source  development  or  production  would  tend  to  eliminate  some 
of  the  arguments  voiced  by  Program  Office  personnel  against  use  of  com¬ 
petition. 
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and  charges  of  unfairness  by  contractors  who  lose,  so  the  source 

selection  must  be  carried  out  in  a  way  which  not  only  chooses  the  best 

design,  but  also  raises  a  minimum  number  of  questions  about  fairness. 

That  is  not  an  easy  task,  particularly  because  little  information  or 

guidance  can  be  drawn  from  the  experience  of  other  programs.  "Lessons 
learned"  reports  from  other  programs  are  rarely  very  useful.  For  the 
most  part,  the  program  manager  must  base  his  planning  solely  on  his  own 
experience.  Some  program  managers  need  no  more,  but  in  other  cases  lack 
of  information  complicates  planning. 

If  awards  are  graiv-^.  to  more  than  one  contractor,  each  additional 
contractor  that  the  Program  Office  must  deal  with  usually  moans  more 
work.  This  is  especially  true  when  cost-type  contracts  are  involved  and 
the  Program  Office  must  monitor  the  costs  of  each  contractor. 

Competition  during  the  production  phase  introduces  management 
complications  of  its  own.  Qualifying  a  second  producer  aftex*  production 
has  begun  can  be  a  major  effort.  It  is  difficult  and  axpensive  to  get  a 
good  technical  data  package  (TOP)  for  the  second  contractor  to  use  in 
starting  production,  and  even  more  difficult  to  persuade  the  first 
producer  to  pass  along  to  a  competitor  the  benefits  of  his  manufacturing 
experience.  The  program  manager  can  choose  to  develop  his  own  TOP,  but 
for  major  programs  this  is  almost  impossible-  Not  all  the  Services  hava 
the  in-house  capability  to  develop  a  TUP,  and  without  this  it  is 
difficult  to  judge  the  adequacy  of  a  TUP.  Sven  with  a  good  TOP,  it 
frequently  takes  a  major  effort  by  the  Program  Office  to  help  the  second 
source  through  all  its  technical  problems  and  into  production.  In  some 
cases,  the  second  source  never  succeeds  in  producing  a  usable  product. 

But  even  in  those  cases,  the  pressure  on  the  first  contractor  may  still 
make  the  effort  worthwhile. * 

Further,  the  Program  Office  oust  work  with  both  contractors  on  such  - 
things  as  quality  control  and  configuration  management.  It  is  generally 
difficult  to  get  two  contractors  to  produce  systems  and  components  with 

*  See  the  discus* sen  of  SRAh  missile  motor  ease  production  in 
Baumbuseh,  at  ii,  Appendixes  to  the  Report  cm  the  Peacetime  Adequacy  of 
the  S-'sver  Tiers  of  the  Defense  Indus tria f  Bases  Case  Studies  of  He  tor 
Systems .  K-*lS-/»-AF,  The  Rand  Corporation,  November  1977 , 
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interchangeable  parts.  If  they  do  not  do  so,  the  Program  Office  faces 
additional  problems  in  spare  parts  procurement.  Further,  each 
additional  production  line  means  an  additional  set  of  non-recurring 
costs  whenever  there  is  an  engineering  change.  Finally,  if  two 
production  lines  are  created,  the  program  manager  must  decide  how  hard 
to  push  each  contractor  in  order  to  ensure  the  benefits  of  competition. 
If  he  pushes  too  hard,  he  runs  the  risk  of  driving  one  of  the 
contractors  out  of  the  program,  and  then  he  is  back  in  a  sole  source 
environment  after  all  the  work  and  expense  of  qualifying  two 
contractors . 

So,  one  factor  in  program  managers'  reluctance  to  introduce 
competition  is  the  perception  that  it  will  make  management  of  their 
program  more  difficult  and  increase  their  workload.  Since  very  few 
program  managers  believe  they  have  enough  well-qualified  people  to  cover 
the  work  of  monitoring  one  source,  they  are  reluctant  to  take  on  even 
more  work  or  to  complicate  matters.  (International  programs  such  as  the 
F-16  add  another  dimension  of  complication,  not  considered  here.) 

It  should  be  noted  that  under  special  circumstances  competition  can 
reduce  the  management  workload  of  a  Program  Office.  If  a  development 
program  is  fixed  price,  and  the  prime  contractor  is  obligated  only  to  a 
"best  effort,"  then  the  program  manager  can  adopt  a  largely  "hands  off" 
management  style,  with  competition  substituting  for  a  host  of 
conventional  Program  Office  management  controls  over  the  contractors. 
However,  this  acquisition  form  is  rarely  used,  the  Services  preferring 
to  retain  considerable  control  over  contractor  actions  even  with  the 
attendant  management  workload.  Another  possibility  is  for  the  pr*m© 
contractor  to  act  as  the  agent  for  the  Government  in  organising 
competition  for  stipulated  subsystems  or  components ,  thus  relieving  the 
Program  Office  of  most  of  the  burden  of  managing  competition. 

the  major  problem  with  having  the  Program  Office  manage  such 
lower -tier  competitions  is  that  they  are  heavily  reguiated.  The  Program 
Office  is  not  as  free  to  deal  with  suppliers  as  the  prime  contractor 
would  be,  and  cannot  be  as  "ruthless"  in  getting  the  best  deal.  In 
addition,  the  regulations  can  slow  down  the  project  and  increase  costs, 
as  well  as  force  the  Program  Office  to  deal  with  suppliers  who  are 


qualified  according  to  the  criteria,  but  are  known  to  be  not  very 
capable.  Also,  some  suppliers  will  not  take  part  in  a  Government- 
conducted  competition  because  they  do  not  like  dealing  with  the 
bureaucracy  and  the  associated  red  tape. 

Once  a  formal  competition  begins,  the  entire  program  is  governed  by 
regulations  that  do  not  apply  in  other  forms  of  acquisition.  This 
results  in  a  reduction  in  management  flexibility  for  the  program 
manager.  For  example,  the  regulations  can  result  in  a  competitive  award 
to  a  prime  contractor  that  the  program  manager  believes  will  not  produce 
a  technically  adequate  product.  The  award  criteria  may  not  always 
screen  out  a  technically  deficient  competitor.  But  in  the  absence  of  a 
formal  competition,  the  program  manager  would  have  more  authority  in 
contractor  selection.  He  may  suspect,  for  example,  that  the  firm  is 
"buying-in'1  by  bidding  lower  than  its  technical  and  economic  resources 
reasonably  justify,  and  is  hoping  to  gat  increased  funding  through  later 
contract  modifications  and  follow-ons.  Or,  he  may  have  enough 
experience  with  or  information  about  the  firm  to  justify  doubts  about 
its  qualifications.  The  other  source  of  constraint  is  that  any  act  that 
affects  one  prime  contractor  must  bo  reflected  in  treatment  of  all  other 
contractors  as  well.  Thus,  tha  program  manager  occasionally  finds 
himself  having  to  operate  in  what  he  views  as  a  non-optiuial  fashion. 

Lack  of  High- Confidence  Near-Torm  Bene f i t  s 

Disincentives  of  the  sort  described  above  tend  to  limit  a  manager's 
enthusiasm  for  introducing  competition  in  his  project.  What  positive 
incentives  exist  to  counterbalance  the  negative  ones?  Apart  from 
exhortations  in  policy  documents  and  the  conventional  wisdom  that 
competition  is  good  for  everyone,  there  are  few  direct  incentives  for 
introducing  competitive  practices.  A  program  manager  is  unlikely  to  be 
rewarded  merely  for  introducing  competition,  heal  cost  reductions  are 
difficult  to  prove  and,  in  recent  experience,  are  likely  to  be  masked  by 
inflation.  Moreover,  given  a  typical  tenure  of  only  about  three  years, 
a  program  manager  is  unlikely  to  be  around  to  receive  the  credit  for  any 
benefits  that  finally  accrue.  The  eosts  of  competition  are  short-term 
and  clear,  while  the  benefits  are  long-term  and  uncertain.  The 


-21- 


■■3 


r. 


incentive  structure  is  more  likely  to  motivate  the  program  manager  to 
look  for  strategies  that  return  short-term  benefits. 

Also,  in  many  cases  competition  is  seen  as  impractical.  There  may 
be  few  qualified  contractors  to  participate  in  a  competition  and  many  of 
those  may  not  wish  to  compete.  Contractors  often  find  that  the 
uncertainties  about  how  a  competition  will  come  out  and  about  the 
criteria  to  be  used  in  the  source  selection  are  sufficient  to  deter  them 
from  entering  a  competition.  Qualifying  a  second  source  can  be  seen  as 
impractical  because  the  production  run  is  too  small,  the  tooling  for  the 
second  production  line  is  too  expensive,  or  the  design  is  too  complex  to 
be  transferable.  In  the  case  of  components,  there  simply  may  not  be 
enough  money  involved  to  justify  the  cost  of  funding  another  source. 

One  commonly  expressed  view  on  competition  is  that  the  potential 
benefits  are  difficult  to  achieve  because  of  the  way  acquisition  is 
currently  conducted.  During  a  development -phase  competition  the  major 
problems  are  perceived  to  be  the  too-early  termination  of  the 
competition.  One  school  holds  that  the  ideal  way  to  conduct  a 
competitive  development  would  be  to  keep  two  contractors  in  the  program 
until  the  design  is  set  and  all  decisions  about  the  production  schedule 
have  been  made.  This  would  allow  the  Program  Office  to  negotiate 
contracts  with  fairly  firm  prices  and  with  production  options  that  keep 
the  prices  fairly  stable  through  the  production  run.  It  also  would 
maintain  the  price  benefits  of  the  competition  after  tho  competition  has 
ended.  Further,  it  would  help  to  set  the  design  while  competitive 
pressure  still  could  be  focused  on  the  contractors,  thus  encouraging 
their  best  dosign  of forts. 

But  that  is  not  the  way  competitions  tend  to  bo  conducted.  Often 
the  competition  ends  before  the  risks  are  rcsolvod  and  tho  production 
contracts  are  fully  set,  with  major  design  changes  land  their  associated 
cost  ineroasos)  still  to  bo  expected.  The  price  benefits  expected  from 
the  competition  may  thus  be  lost.  If  design  changes  are  made  after 
competition  has  onded,  either  to  dual  with  residual  technical  problems 
or  to  add  capabilities,  then  any  contracts  will  have  to  be  renegotiated 
in  a  sole  source  environment.  Prime  contractors  and  sub-contractors 
will  be  less  willing  to  accept  low  pricos  in  a  sole  source  environment 
than  in  a  competitive  environment. 


-22- 


OVERVIEW 

Thus  far,  this  section  has  been  devoted  to  a  review  of  problems  and 
disincentives  regarding  the  use  of  competition  in  the  acquisition  of 
major  weapon  systems.  At  this  point  we  turn  to  a  somewhat  more 
speculative  attempt  to  synthesize  that  information  and  put  it  into  a 
broader  context  so  that  we  can  interpret  it  and  suggest  some  remedies. 
Because  of  limited  and  incomplete  information  the  following  ideas  must 
be  characterized  as  hypotheses  rather  than  conclusions. 

We  first  observe  that  managers  seem  quite  willing  to  employ 
competition  among  firms  prior  to  the  beginning  of  full-scale 
development,  but  generally  are  reluctant  to  establish  price  competition 
during  production.  The  reasons  seem  rather  obvious:  each  is  the  path  of 
least  resistance.  The  idea  of  receiving  proposals  and  bids  on  a  new 
item  from  multiple  competitors  is  firmly  and  deeply  engrained  in  our 
institutions,  and  it  is  relatively  inexpensive.  Despite  all  the  work 
involved  in  preparing  bid  packages  and  performing  source  selections,  it 
is  probably  easier  than  obtaining  approval  for  sole-source  negotiations, 
and  that  becomes  more  and  more  true  as  the  size  of  the  project 
increases.  Note  that  we  are  here  talking  about  competition  before  the 
beginning  of  full-scale  development,  where  the  selection  criteria 
involve  some  subtle  blend  of  design  concept,  perceptions  of  risk,  and 
estimates  of  system  performance,  delivery  schedule,  and  cost.  While  the 
formal  source  selection  board  will  usually  establish  relative  weights 
for  thoso  and  other  criteria,  the  final  selection  is  actually  performed 
by  a  few  senior  officials  and  is  based  on  a  highly  intuitive  blend  of 
such  factors.  Although  the  benefits  are  not  easily  measured,  most 
managers  are  comfortable  with  this  process  and  are  convinced  that  such 
competition  produces  a  "better"  product  bocause  it  encourages  each 
competitor  to  use  his  best  people  and  to  work  very  hard. 

In  contrast,  consider  price  competition  during  the  procurement 
phase.  Horo  the  problems  and  disincentives  described  earlier  loom 
relatively  largo;  the  introduction  of  competition  may  significantly 
complicate  program  management  tasks  in  a  variety  of  ways.  These 
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bureaucratic  problems  can  be  extremely  subtle  and  complex,  and  the 
present  study  has  been  able  to  barely  scratch  the  surface.  We  are 
persuaded,  however,  that  such  problems  play  a  significant  role  in  the 
extent  and  success  of  competition  in  weapon  acquisition  and  that  they 
deserve  more  careful  analyisis. 

These  institutional  disincentives  are  compounded  by  the  fact  that 
managers  have  cause  to  be  genuinely  skeptical  about  the  payoffs  to  be 
obtained.  This  skepticism  takes  three  forms: 

o  Doubts  as  to  the  evidence  that  any  significant  savings  were  in 
fact  obtained  when  price  competition  was  introduced  in  previous 
reprocurement  actions. 

o  Even  if  there  is  some  evidence  that  savings  were  achieved  in 
the  past,  doubts  as  to  the  likelihood  that  savings  can  be 
achieved  "now"  if  price  competition  is  introduced  for  the 
particular  procurement  action  being  considered  In  other 
words,  skepticism  about  the  guidance  available  for  identifying 
those  procurements  for  which  competition  is  liKely  to  yield 
significant  net  dollar  savings.  The  more  sophisticated  the 
executive- -the  more  he  recognizes  both  the  costs  of  introducing 
competition  and  the  significance  of  discounting--the  more 
skeptical  he  is  likely  to  be. 

o  Even  where  savings  or  other  benefits  seem  achievable,  the 

program  manager  often  doubts  that  the  proposal  for  competition 
can  be  sustained  through  the  many  layers  of  the  management 
structure,  especially  because  the  required  front-end  funding 
may  be  competing  with  another  program's  very  survival. 

If  this  broad  hypothesis  is  true  (and  we  believe  the  evidence 
supports  it),  then  two  implications  are  clear: 

1.  Little  or  no  additional  incentive  is  needed  to  encourage  active 
competition  during  the  concept  formulation  and  the 
demonstration  and  validation  phases  of  a  new  weapon  system. 
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2.  Managers  would  have  stronger  incentives  to  employ  price 

competition  during  the  procurement  phase  (together  with  the 
necessary  investment  during  hardware  development  and  testing) 
if  they  had  (a)  evidence  that  it  will  in  fact  yield  net  savings 
(after  accounting  for  all  costs),  (b)  practical  guidance  on  how 
to  select  promising  candidates  for  price  competition  and  how  to 
structure  an  acquisition  strategy  to  best  achieve  expected 
savings,  and  (c)  active  support  for  the  front-end  funding  of 
competitive  actions  within  the  budgetary  process.  Provided 
with  such  information  and  support,  managers  may  find  price- 
competitive  procurements  more  attractive.  Without  such 
information,  the  more  venturesome  managers  may  try  to  introduce 
competition  when  their  intuition  and  judgment  suggest  it  might 
pay  off,  but  cautious  managers  may  find  the  obstacles 
insuperable. 


III.  PRICE  COMPETITION  PUKING  RKPKOCl'KKMKNT 


Omi  of  the  disincentives  to  the  use  of  competition  in  the 
development  phase  is  that  the  costs  are  near-term  and  tangible,  while 
the  benefits  are  long-term  and  difficult  to  identify  or  measure.  It  is 
the  mixture  of  design  and  price  competition,  characteristic  of 
competitive  development,  that  makes  it  so  difficult  to  identify  and 
measure  benefits.  To  measure  the  benefits  of  competitive  development 
involves  comparing  the  price  and  tjig  quality  or  performance  of  the 
winning  system  with  the  price  and  the  quality  or  performance  of  the 
system  that  would  have  gone  into  production  had  there  been  no 
competition.  It  is  difficult  enough,  as  the  research  reviewed  in  this 
section  demonstrates ,  to  make  a  quantitative  comparison  on  price  alone; 
if  quality  and  performance  are  also  to  be  taken  into  account,  the  basis 
for  comparison  will  be  largely  qualitative  and  possibly  quite 
subjective. 

If  competition  is  introduced  in  th*  production  phase,  costs  3re 
again  near-term  and  tangible,  while  benefits  are  in  the  future.  But  the 
future  benefits  are--or  at  least  should  be- -somewhat  easier  to  identify 
and  measure  because  competitive  production  is  usually  price-competitive. 
If  price  competition  is  introduced  after  there  has  been  sufficient  sole- 
source  production  to  establish  a  learning  curve,1  then  any  difference 
between  the  post -competition  price  and  the  anticipated  sole-source  price 
for  the  same  quantities  can  reasonably  be  attributed  to  competition, 
assuming  that  other  aspects  of  the  program  remain  reasonably  constant 
or  that  their  effects  on  price  can  be  estimated.  This  is  an  important 
assumption  and  one  that  does  not  always  accord  with  the  realities  of 
competitive  procurement. 

1  A  learning  curve  is  a  graphical  repress***  at  ion  of  the  relation¬ 
ship  between  the  cost  of  producing  an  extra  unit  and  the  cumulative 
quantity  already  produced.  Such  curves  usually  have  a  negative  slope; 
i.e. .  the  cost  of  producing,  say,  the  200th  item  is  lass  than  the  tost 
of  producing  the  100th  item,  learning  curves  are  usually  graphed  on  a 
log- log  grid  because  in  logarithmic  form  the  curve  can  be  represented  as 
a  straight  line. 
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Competition  is  sometimes  introduced  in  the  production  phase  for 
reasons  having  nothing  to  do  with  price;  the  decision  to  reprocure  a 
system  competitively  may  be  motivated  by  a  desire  to  improve  performance 
or  contractor  responsiveness,  viz.  the  Sparrow  AIM-7F.2  Even  when  the 
primary  motive  for  turning  to  competitive  reprocurement  is  an  interest 
in  lowering  prices  paid  by  the  Government  buyer,  the  introduction  of 
competition  can  affect  things  other  than  price.  In  particular,  the 
change  from  sole-source  to  competitive  reprocurement  may  affect  both  the 
delivery  schedule  and  the  performance  or  quality  of  the  items  delivered. 

Quality  and  delivery  schedule  are  critical  to  system  effectiveness, 
although  analysts  making  cost  comparisons  between  alternative  system 
procurements  have  usually  assumed  that  each  procurement  provided  the 
same  or  very  nearly  the  same  item  at  the  same  or  very  nearly  the  same 
time.  The  real  world  is  seldom  that  neat.  In  principle  it  is  possible 
to  adjust  savings  estimates  to  reflect  differences  in  quality  and 
delivery  schedule;  in  practice,  however,  it  is  very  difficult.  The 
problems  are  similar  to  those  involved  in  evaluating  the  benefits  of 
development  competition, 

Thus,  while  there  are  opportunities  to  measure  the  benefits  of 
competition  in  the  production  phase,  these  opportunities  are,  for  the 
most  part,  limited  to  measuring  the  dollar  belief its  of  competition 
in  the  reurocuvcmcnt  ghasg.  These  effects  are  the  roost  quantifiable  of 
all  the  effects  of  competition;  nevertheless,  it  is  not  a  simple  task  to 
develop  reliable  estimates  of  these  effects. 

The  need  to  predict  the  price  reductions  attainable  from 
competition  derives  free  the  fact  that  (for  other  than  off-the-shelf 
items)  there  usually  are  substantial  additional  costs  cssoeiated  with 
introducing  additional  producers.  These  include  the  costs  of 
transferring  production  technology;  the  costs  of  Technical  Data  Packages 
(T8Ps),  technical  assistance,  learning  buys,  and,  not  infrequently, 
subsequent  claims  against  the  Government  for  faulty  TOP*  or  other 
inadequacies  in  the  technology  transfer  process.  These  technology 
transfer  costs  are  usually  borne  by  the  Government,  Additional  start-up 


bee  iiilv.  Gates,  and  Sthuttinga  (1DA#9).  App.  8. 


costs  are  often,  but  not  always,*  incurred  by  the  Government  buyer 
when  a  given  quantity  of  an  item  is  produced  by  more  than  one  firm. 

These  additional  start  t>  costs  include  the  duplication  of  tooling, 
overhead  costs,  and  work-force  learning. k 

There  also  are  added  internal  costs  to  the  Government  in  conducting 
the  competitive  selection  process  (advertising,  briefings,  evaluating 
bids,  etc.)  and  in  managing  a  competitive  production  environment.  It 
may  sound  contradictory  to  speak  of  ''managing"  a  competitive  production 
environment,  given  that  one  of  the  purported  advantages  of  fixed-price 
competitive  awards  is  the  reduction  of  DoD's  need  to  monitor  costs 
incurred  during  the  production  process.  While  costs  and  the  correct 
application  of  profit  weights  to  various  factors  of  production  need  net 
be  monitored,  Program  Offices  handling  a  competitive  reprocurement  may 
nevertheless  discover--to  their  chagrin- -other  managerial  tasks  needing 
attention.  These  include  second  producers  that  default  on  their 
contracts  without  ever  delivering  an  item,  as  happened  twice,  for 
example,  during  the  Army's  attempt  to  reprocure  the  PP-4763/GRC  power 
supply  competitively  (APRQ78,  pp,  56-56) ,  Managerial  problems  also 
include  companies  which,  after  winning  &  competitive  award,  fail  to 
deliver  on  schedule  and  subsequently  blame  the  Government  buyer  both  for 
the  delays  in  the  delivery  schedule  and  company  losses  on  the  contract. 
For  example,  in  the  case  of  the  PRC-77  radio  set,  a  company  submitted  a 
claim  for  $10,721,728  alleging  "increased  costs  for  everything  from 
specific  design  defects  to  lost  profits  due  to  the  Mexican  Peso 
devaluation  in  September  1976"  [APK078,  p.  40).  Thus  the  managerial 

s  If  the  quantity  per  month  to  be  reprocured  is  sufficiently 
greater  than  the  quantity  per  month  procured  under  the  original  sole- 
source  contract  so  that  a  second  production  line  would  have  to  be  set  up 
anyway,  then  most  of  these  additional  start-up  costs  would  have  to  be 
incurred  with  or  without  the  introduction  of  competition  and  should  not 
be  debited  from  the  savings  attributable  to  competition. 

Thu  effect  of  work-force  learning,  and  other  factors  reflected 
rn  learning  curves,  on  price  reductions  attainable  from  competitive 
^procurement  has  received  considerable  attention.  In  fact,  as  we  point 
out  later,  reliance  on  product  ion -cost  learning  curve  theory  as  a  key 
explanatory  model  when  examining  competitive  r«procu cement  appears  to 
have  been  excessive.  Rather  than  the  theory  explaining  the  findings; 
more  often  than  not,  the  findings  contradict  the  theory. 
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burden  experienced  by  a  Program  Office  is  not  always  eased  us  a  result 
of  having  reprocured  an  item  competitively. 

For  competition  to  have  a  net  dollar  payoff,  the  sura  of  all  these 
costs  must  (at  least)  be  offset  by  price  reductions  induced  by  the 
competitive  process.  It  is  not  enough  simply  to  have  faith  that  the 
competitive  process  will  lower  prices  ay  some  amount.  A  forecast  of 
that  amount  is  needed  in  order  to  judge  whether  gross  savings  are  likely 
to  be  sufficiently  greater  than  the  costs  of  opening  additional  sources 
of  supply  to  justify  both  the  costs  and  the  risks  (of  performance  problems 
and  delayed  delivery)  associated  with  competitive  reprocuroment . 

In  this  section  wo  (a)  review  the  findings  and  conclusions  reported 
in  past  studies  of  competitive  voprocureraent ,  focusing  on  work  done  for 
or  within  DoD  in  the  1970s;  (b)  examine  some  cf  the  problems  involved  in 
estimating  the  dollar  benefits  realized  fx'ora  competitive  reprocurement 
in  the  past  and  in  predicting  the  price  reductions  that  might  be 
attained  from  the  use  of  competitive  raproeureraent  strategies  under 
varying  conditions  in  the  future;  and  (c)  discuss  some  of  the 
implications  of  these  findings  and  problems. 

We  focus  most  of  our  attention  on  four  relatively  comprehensive 
studies  done  for  or  within  DoD;  a  1972  study  by  the  Q.8.  Arsxy 
Electronics  Copland  CEGQM72),  a  1974  study  by  the  Institute  fur  Defense 
Analyses  ( JDA74),  a  19?§  study  by  the  l*.S-  Army  Procurement  f'*@areh 
Office  (APK078),  and  a  second,  more  recent  (1979)  study  by  the  Institute 
for  Defense  Analyses  (IBA'f). 

THE  FOUR  KEY  STlTHESi  EC6HT2.  IDA74,  &PK07S,  and  I8A79 

Quantitative  research  in  the  1970s  concentrated  on  several  distinct 
tasks,  One  is  a  statistical  task  which  involves  developing  a  model  to 
predict  price  changes  attributable  to  competition.  EC&H72  was 
interested  only  in  this  predictive  task;  the  three  more  retent  studies 
tackled  other  tasks  as  well.  Chief  ssong  these  other  tasks  in  our 
opinion- » a  1 though  not  necessarily  in  the  opinion  of  the  authors  whose 
work  we  review-is  what  we  shall  refer  to  as  the  accounting  task.  The 
accounting  task  concerns  itself  with  "the  bottos  line;**  whe.t  is  the 
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appropriately  discounted  not  dollar  payoff*  from  an  iuvestaeat  made  to 
introduce  competition  in  tha  reprocuroment  phase? 

Only  APK07S  and  IDA79  addressed  the  accounting  task  explicitly. 
APR078  emphasized  the  importance  of  including  the  costs  incurred  by  the 
Government  buyer  before,  during,  and  after  the  introduction  of 
competitive  reprocuremunt .  i.e.,  the  importance  of  examining  net  rather 
than  gross  savings.  IDA79,  on  the  other  hand,  emphasized  the  importance 
of  discounting,  i.e.,  c£  taking  into  account  the  time  value  of  money  by 
estimating  the  net  present  value  of  and/or  the  internal  rate  of  return 
from  investments  made  to  introduce  competition  during  reprocurement . * 

We  introduce  each  study  by  describing  its  objectives,  the 
’sample*7  of  competitively  reprocured  items  analyzed,  and  the  key 
findings  and  conclusions  related  to  price  competition  in  the 
reprocurement  phase. 

The  First  Study.  EC0H72 

This  study  by  the  Cose  Analysis  Division  of  the  S"-2H  Comptroller's 
Office  set  out  “to  determine  if  a  predictive  model  or  aetbedology  could 
be  established"!?,  ij  to  forecast  uaifprics  reductions  from  competitive 
reprocuresent , 

The  I60H?2  analysts,  limiting  their  search  to  electronic  items 
managed  by  ES0?!,  were  able  to  find  prins  data  i»  22'  instances  of 
competitive  reprocurement.  They  found  data  on  quantities,  lead  times* 


1  The  savings  that  result  t?§a  g@«p@tit.l@n  accrue  ever  time' 
sometimes  many  years  removed  from  the  investment-  itesee,  discounting  to 
net  dollars  is  needed  so  iniiccte  the  true  economic  coosutae&r**  of  the 
investment  - 

6  @n  discounting,  net  present  value*  and  internal  rates  of  re* 

turn,  see  fisher  £1§?©L  gp.,  III  et  seq.*  Mirshleifer  itcSeas 

CitiS);  and  iteishks 

7  The  word  *  sample*  is  enclosed  in  single  quetes  throughout  to 
signal  that  these  are  not  samples  In  a  formal  statistical  sense--  This 
means  that  It  is  difficult  t»  make  inferences  snout  ether  cespetltive 
reprecurements  from  the  results  obtained  fey  analysing  these  potentially 
nonarep?esentative  instances  of  eempetitive  reprosvrement. 

Throughout  this  section,  we  also  distinguish  between  the  'samples* 
used  ter  statistical  analyses  and  the  existing  data  base-  hash  study 
collected  data  on  Items  which  were  net,  for  one  reason  ot  taertat,  is* 
eluded  in  its  ’sample  *  Such  items  are  sometimes  picked  up  fey  a  sufeee* 
quest  study  and  included  in  its  'sample,*  boo*  for  instance*  Table  2, 
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and  delivery  schedules --the  predictor  variables  of  interest  to  EC0M72-- 
for  only  13  of  the  22  cases.  Table  2  lists  these  22  items,  along  with 
their  sole-source  and  post-competition  unit-prices,  and  indicates  which 
of  the  items  were  used  in  the  ECOM72  'sample'  and  which  have  been  used 
in  subsequent  'samples.'  These  items  first  entered  production  between  FY 
1958  and  FY  1967. 

ECOM72  looked  at  unit-price  reductions  on,  in  almost  every  case, 
the  first  competitive  buy.  All  ECQM72  first  buys  were  buy-outs.* 

The  reported  arithmetic  mean  of  unit-price  reductions,  not  adjusted  for 
inflation  or  learning  effects,8 9  was  54  percent  on  the  data  base  of 
22  items,  with  a  standard  deviation  of  15  and  a  range  from  12  to  78  percent. 

On  the  'sample'  of  13  items  used  in  EC0M72's  attempt  to  develop  a 
"predictive  model  or  methodology s "  a  mean  unit-price  reduction  of  53 
percent  is  reported.  ECOM72  reported  unit-price  reductions  on 
individual  items  only  for  this  subset  of  13;  Table  3  ranks  these  items 
by  unit-price  reduction. 

The  ECOM72  analysts  used  multiple  regression  in  their  attempt  to 
develop  a  model  that  would  predicc  unit-price  reductions.  They  used 
three  ratios  as  predictor  variables:  competitive  lead  time  over  sole- 
source  lead  time,  competitive  quantity  over  sole-source  quantity,  and 
competitive  delivery  vate  (quantity  per  month)  over  sole-source  delivery 
rate.  They  concluded  that  none  of  the  regression  equations  examined 
were  good  candidates  for  a  predictive  model,  and  that 

To  further  pursue  the  attempts  of  finding  more  significant 
causal  relationships  among  luad  tiro®,  quantity,  and  delivery 
by  regression  techniques  appears  futile.  It  can  almost  be 
concluded  that  the  desired  relationship  is  severely  clouded  by 
the  other  variables  that  would  be  difficult  to  quantify.  Also, 

8  We  use  the  term  "buy-out"  to  refer  to  awards  that  go  to  one 
producer,  i.e.,  to  distinguish  from  a  split-buy.  The  term  "winner- 
take-all"  will  be  used  to  refer  to  a  buy-out  that  is  intended  to  inehid* 
all  future  quantities  of  the  item. 

9  IDA79  derived  learning  curves  from  prices  on  those  EUOM72  items 
where  there  were  at  least  two  sole-source  buys  and  found  the  curves  to 
be  so  shallow  that  adjustment  for  sole-source  learning  has  little  effect 
on  mean  unit -price  reduction,  {p.  52) 


Table  2 


ITEMS  IN  THE  ECOM72  DATA  BASE  BY  UNIT  PRICES  AND  USE  IN  SAMPLES 


|  ! Sole-Source  j  Post- 

|  Used  in  Which  'Sample'?  jUnit  Price  |  Competi- 

Items  in  the  ECOM72  j _ |Used  by  j  tion  Unit 

Data  |  |EC0M72(a)  ■  Price(b) 

1 EC0M72 1 IDA74 1 APR078 1 IDA79  j _ j _ 

!  |  i  I  I  (then-year  $) 


1.  AN/ARC- 34  Airborne 


Radio  Set 

X 

X 

5620. 

1381. 

*) 

**  • 

Squad  Radio:  AN/PRT-4 

X 

X 

290. 

(c)  170. 

AN/PRR-9 

3. 

TD-352  Multiplexer 

X 

X 

9330. 

3653 

4. 

TD-202  Radio  Combiner 

X 

X 

4710. 

1717 

5. 

TD-204  Cable  Combiner 

X 

X 

4951 . 

1655 

6. 

TD-660  Multiplexer 

X 

X 

7931. 

3234 

7. 

MD-522A/GRC  Modulator- 

Demodulator 

X 

X 

X 

2960. 

1275. 

A 

C  . 

AN/ARC-131  Radio 

X 

2.975. 

2132. 

4, 

AN/l'PM-96  Test  Set 

X 

X 

9130. 

5150. 

10. 

PP-4763/GRC 

X 

1095. 

461. 

11. 

MK-980/PPS-5 

X 

X 

10658. 

3718 . 

12. 

AN/APM-123  Test  Sot 

X 

X 

6350. 

2078. 

13. 

AN/GRC-1Q3  Radio  Sot 

X 

X 

.  24404. 

9995. 

14. 

AN/SRC-106  Radio  Sot 

X 

X 

15137. 

6897 . 

13, 

AN/PRC-77  Radio  Sot 

X 

X 

X 

937. 

487. 

16. 

AN/ PRC -25 

2157. 

843. 

17. 

TD-206/6 

460, 

100. 

16, 

AN/FLR-9 

Cd) 

Cd) 

19. 

.  S-2S0/6  Shelter 

X 

3826. 

1827, 

20. 

AS/FYC-8X 

12630. 

7110. 

21. 

CV-1346  Signal  Convertor 

X 

X 

X 

3S46, 

144Q. 

22. 

AN/ASK-43 

X 

205S. 

1800. 

SOURCES:  EC0M72.  pp.  7-S 

,  Apps.  1  and 

nit  ida?4, 

p.  H-3; 

APR078, 

£>* 

b6;  1DA?9.  pp.  A-4. 

(a)  Sale-source  prices  us^d 

by  ECOM72 

are 

not  adjusted  for 

learning 

effects,  Nov  are  they  always 

the  prices  from 

the  last 

sole-source  buy 

before  competition. 

(b)  P-ye  ite&is  used  in  the  ECOM72  ’sample,*  post -competition  prices 
reported  oy  ECOH72  on  p.  A- 1  are  used.  For  other  items,  the  price  or 
average  of  prices  reported  on  pp.  C-i  to  C-4  are  used. 

(c)  The  average  of  the  two  sole-source  prices  which  ECOM72  indicates 
were  used  (p.  C-U  is  £8b. 

(d)  Not  available. 
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Table  3 
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ITEMS  IN  THE  ECOM72  SAMPLE  WITH  UNIT  PRICE 
ON  FIRST  COMPETITIVE  AWARD,  ORDERED  BY  UNIT-PRICE  REDUCTION 


«  "t 

<i 


Unit -Price  Reduction 
(not  adjusted  for 
inflation  or  sole- 
: source  learning) 
(percent) 


Unit-Price, 
First  Compet¬ 
itive  Buy 
(then -year  $) 


1. 

AN/ARC-54  Airborne  Radio  Set 

75. 

1381. 

2. 

AN/APM-123  Test  Set 

67. 

2078, 

3. 

MK-980/PPS-5 

65. 

3718. 

4. 

CV-1548 

59 

1440. 

5. 

AN/CRC-103  Radio.  Set 

59. 

9995 . 

6 . 

PP-4763Q/GRC 

58. 

461. 

*/ 

MD-522A/GRC 

57. 

1275. 

8. 

AN/GRC-106  Radio  Set 

54. 

6897. 

9. 

S-25C  Shelter 

52. 

1827. 

10. 

AN/PRC-77  Radio  Sot 

48. 

487. 

*1. 

AN/UPM-9SQ 

44, 

5150. 

12. 

Squad  Radio:  AN/FRT-4 

AN/PKR-9 

41. 

170. 

13. 

AN/ASN-43Q 

12, 

1800. 

Arithmetic  Mean 

=  S3. 
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the  number  of  .  .  .  items  making  the  transition  from  sole-source 
to  competition  each  year  is  small,  and  to  accumulate  a  large 
enough  sample  to  provide  sufficient  degrees  of  freedom  if  the 
variable  list  were  expanded  would  span  many  years,  [p.  22] 

The  sole  objective  of  this  study  was  to  determine  if  a  predictive  model 
could  be  developed.  Having  decided  that  it  would  be  very  difficult,  if 
not  impossible,  to  develop  such  a  model,  the  authors  state  in  the.tr 
closing  paragraph: 


The  only  conclusions  that  can  be  drawn  from  this  study  are 
that  unit  price  is  substantially  reduced  by  cempetition-- 
possibly  more  than  was  thought--and  that  the  difficulties  that, 
might  be  voiced  and  thought  to  exist  when  dealing  with  the 
problems  encountered  by  the  new  awardee  are  well  worth  the  ef¬ 
fort  in  dollars  saved.  Also,  use  of  the  25-30™  reduction  for 
planning  purposes  appears  very  conservative.  Larger  reduc¬ 
tions  for  systems  consisting  of  a  good  mix  of  major  items 
could  approach  40-50%  with  some  degree  of  confidence  since 
very  few  occurrences  were  noted  below  this  range,  [p.  24] 


The  Second  Study:  IDA74 

This  study  by  Zusman  et  al.  was  conducted  for  the  Advanced  Reseach 
Projects  Agency  (ARPA).  It  included  three  tasks,  only  one  of  which 
concerns  us  here:  an  examination  of  competitive  reprocurement  "to  measure 
quantitatively  the  effect  of  competition  on  selling  price."  (p.  45] 

The  IDA74  team  contacted  22  principal  agencies  within  DoD,  and  47 
divisions  or  offices  within  thoso  agencies,  in  thoir  soareh  for  data--a 
search  that  turned  up  usable  data  on  only  19  items. ie  The  criteria  for 
inclusion  of  an  item  in  the  IDA74  ‘sample*  were  not  overly  demanding: 
"retrievable  price  (not  cost)  data,"  "at  least  two  solo-source 
production  awards,"  "»t  least  one  competitive  award,"  and  a  "unit  eost 
of  at  least  S1CJOO."  While  there  are  only  19  hardware  items  in  the  1DA74 
‘sample,'  the  authors  used  20  instances  of  competitive  reprocurement  in 
their  analysis.  They  used  the  Sullpup  twice:  as  a  competitive  split-buy 
won  by  Martin  and  as  a  winner-take-ail  award  won  by  Haxson  (See  Tablu  4) 

..  search  is  described  iu  !DA74*s  App.  G. 
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Table  4 

ITEMS  IN  THE  IDA74  SAMPLE  WITH  UNIT  PRICE 
ON  FIRST  COMPETITIVE  AWARD,  ORDERED  BY  UNIT-PRICE  REDUCTION 


Item 

Unit-Price 

Reduction 

(percent) 

Unit-Price,  First 
Competitive  Buy (a) 

(1970  dollars) 

1. 

MD-522  Modulator-Demodulator 

60.3 

1275. 

2. 

TD-352  Multiplexer 

57.8 

4291. 

3. 

Aerno  60-6402 

57.0 

3030. 

4. 

CV-1548  Signal  Converter 

53.7 

1503. 

5. 

MK-48  Warhead 

S3. 2 

5087. 

6. 

TD-202  Radio  Combiner 

52.5 

1741. 

7. 

TD-204  Cable  Combiner 

50.2 

1877. 

8. 

TOW  Missile 

48.1 

1999. 

9. 

Bullpup  (Maxson) 

45.8 

1474. 

10. 

Talos  Guidance  and  Control 

42.3 

87636. 

11. 

MK-48  Electric  Assembly 

37.5 

6027. 

12. 

USM-181  Telephone  Test  Set 

36.0 

422. 

13. 

APX-72  Airborne  Transponder 

32.6 

1653. 

14. 

FGC-20  Teletype  Set. 

32.0 

1308. 

15. 

TD-660  Multiplexer 

30.2(b) 

3524. 

16. 

SPA-25  Radar  Indicator 

21.3 

6819. 

17. 

Bullpup  (Martin) 

13.8(b) 

3725. 

18, 

Hawk  Metal  Motor  Parts 

6.4 

1014. 

19, 

Rockeyc  Cluster  Bomb 

5.3 

1641, 

20. 

Shillelagh  Missile 

-0.2 

3041. 

Arithmetic  Mean  «  36.8 

SOURCE;  IQA74,  Table  10,  p.  56  and  Table  H-0.  p.  li-3 

(a)  More  or  less  the  first  competitive  buy;  sou  text. 

(b)  Last  digit  not  clear  in  our  copy  of  IUA74. 


These  20  instances  include  six  items  from  the  ECOM72  data  base,  two 
of  which  were  used  in  EC0M72's  regression  analysis.  Most  of  the  items 
in  IDA74's  'sample'  were  procured  by  the  Army. 

IDA74  examined  unit-price  reductions  on  the  first  competitive  buy, 
as  did  ECOM72,  but  for  the  TOW  and  the  second  Bullpup  c.,se  (Maxson) ,  the 
first  buy-out  following  competitive  dual-sourcing  was  used.  For  the 
first  Bullpup  case  (Martin),  the  Rockeye  cluster  bomb  and  the 
Shillelagh,  the  first  competitive  buy  used  is  a  competitive  split-buy. 

IDA74  used  constant  (1970)  dollars  to  control  for  inflation  and 
adjusted  for  sole-source  learning  effects  by  extrapolating  an  expected 
sole-source  price  from  a  learning  curve  based  on  unit  prices . 1 1  The 
arithmetic  mean  of  the  unit-price  reductions  on  the  first  competitive 
buy  as  defined  by  IDA74  is,  for  this  group  of  mostly  Army  electronic 
items  and  missiles,  36.8  percent.  The  median  is  39.9  percent. 

IDA74  used  multiple  regression  analysis  to  examine  the  effect  on 
post -competition  unit-price  reductions  of  four  variables:  the  exponent 
of  the  sole-source  learning  curve,  the  ratio  of  competitive  to  sole- 
source  quantities,  "type  of  competition,"  and  number  of  bidders  as  a 
measure  of  "the  intensity  of  competition."  The  effect  of  the  number  of 
bidders  was  not  statistically  significant.  1DA74  does  not  proffer  the 
regression  equation  derived  as  a  predictive  tool;  the  authors  merely 

*  * 1  The  derivation  of  sole-source  learning  curves  from 
price/quantity  data  points,  rather  than  production-cost/quantity  data 
points,  is  the  rule  rather  than  the  exception  in  studies  of  competitive 
roprocuremont .  IDA74  offers  the  following  justification  for  this  prac¬ 
tice: 


While  progress  (production-cost  loarning  curve)  theory  is 
based  on  a  relationship  between  cost  and  quantity,  it  was 
found  that  the  only  data  available  to  us  were  price  data. 

Under  a  sole-source  contract,  where  the  government  is  monitor¬ 
ing  the  contractor's  costs,  marginal  costs  and  marginal  prices 
are  highly  correlated;  and  it  is  expected  that  progress  theory 
will  accurately  reflect  price  behavior.  Therefore,  wo  have 
referred  to  the  sole-source  price-quantity  relationships  as 
progress  curves,  (Emphasis  in  the  original.) 

Among  the  four  studies  we  focus  on,  only  APR078  used  production  costs 

rather  than  prices  in  estimating  learning  curves. 
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summarize  the  nature  of  the  three  statistically  significant 
relationships.  (1)  "The  steeper  the  progress -curve  slope  the  less  that 
is  likely  to  be  saved."  (2)  On  the  first  competitive  buy,  maximum 
savings  are  achieved  with  a  buy-out  competition,  minimum  saving?  with  a 
50/50  split-buy  (each  competitor  gets  half  of  the  total).  The  authors 
caution  that  if  "there  are  additional  buys  after  the  first  competitive 
buy,  then  a  (buy-out)  strategy  may  not  be  optimum."  (3)  Unit -price 
reduction  is  negatively  correlated  "with  the  ratio  of  the  number  of 
units  bought  under  the  first  competitive  award  to  the  total  number  of 
units  produced  under  all  the  sole-source  awards."  [p.  59] 

A  negative  correlation  between  unit-price  reduction  and  the  ratio 
of  the  first  competitive  quantity  to  sole-source  quantity  was  also 
reported  by  ECOM72.  IDA74's  replication  of  this  finding  on  a  more 
varied  set  of  items  is  interesting  given  that  production-cost  learning 
curve  theory,  conventional  wisdom,  and  the  DAR12  lead  one  to  expect  a 
positive  correlation. 

The  Third  Study:  APR078 

Lovett  and  Norton  of  the  U.S.  Army  Procurement  Research  Office 
wanted  to  "(i)  develop  a  methodology  to  estimate  the  net  savings 
achieved  due  to  competition,  (ii)  further  develop  the  methodology  to 
forecast  the  net  savings  expected  from  introducing  competition  into  the 
procurement  of  future  major  weapons  systems,  (iii)  furnish  an  organized 
data  base  to  support  the  net  savings  methodologies."  [p.  ii] 

The  APR078  'sample'  consists  of  16  Army  and  Navy  systems;  three  of 
those  were  also  in  the  IDA74  'sample'  (the  Shillelagh,  TOW,  and  Bullpup 
missiles)  and  two  were  in  the  F.COM72  'sample'  (the  AN/ARC-131  and 
AN/PRC-77  radios). 

~  H  Soction  3-106  on  the  Negotiation  of  Initial  Production  Con¬ 
tracts,  Paragraph  (d)  states: 

The  numbs r  of  items  to  be  procured  under  an  initial  production 
contract  will  be  established  only  after  considering  all  por- 
tinont  factors,  including  the  practical  minimum  quantity  suit¬ 
able  to  permit  the  development  of  the  production  design  and  a 
data  package  adequate  to  establish  competitive  procurement  of 
the  item  at  the  earliest  practicable  date.  (Emphasis  added.) 
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APR078  was  the  only  study  to  collect  production-cost  data.  For 
systems  on  which  such  data  were  available,  sole-source  learning  curves 
were  derived  from  disaggregations  of  and  adjustments  to  those  data.  To 
arrive  at  net  savings,  the  APR078  analysts  also  collected  all  the  data 
they  could  find  on  the  external  costs  of  introducing  competition.  Those 
were  primarily  costs  related  to  the  transfer  of  production  technology 
from  one  firm  to  another;  technical  assistance  and  learning  buys  for  the 
second  source  are  included,  but  not  the  cost  of  TDPs.  Successful  claims 
against  the  Government  by  the  second  source  for  faulty  TDPs  and  the  like 
are  also  included.  Due  to  time  constraints,  the  APR078  analysts  did  not 
attempt  to  collect  data  on  the  in-house  costs  of  competition,  i.e.,  the 
additional  expenses  incurred  by  the  Government  buyer  in  setting  up  and 
managing  a  competitive  environment.  Thus  their  estimates  of  actual 
savings  and  losses  are  estimates  of  partly  netted  savings  and  losses. 

These  estimates  of  savings  and  losses  were  not .discounted,  nor  were 
EC0M72*s  and  IDA74*s  price  reductions.  They  were  savings  or  losses  over 
all  quantities  after  the  first  buy-out  competition,  including  expected 
future  buys.  (Recall  that  IDA74's  price  reductions  were  on  the  first 
competitive  buy  only,  whether  that  was  a  split-buy  or  a  buy-out. 

ECOM72's  were  mainly  on  the  first  competitive  buy,  and  all  instances 
were  buy-outs.)  APR078  reported,  in  FY  1972  dollars,  these  partly  netted 
savings  and  losses  as  a  percentage  of  what  total  procurement  costs  would 
have  been  if  all  procurement  had  been  sole  source. 

These  partly  netted  savings  and  losses  as  a  percentage  of  total 
procurement  costs  are  shown  in  Table  5. 

APR078  also  computed  average  percentage  unit-price  reduction  across 
all  post-competitive-buy-out  quantities,  including,  as  in  the  estimate 
of  partly  netted  savings,  quantities  budgeted  for  but  not  yet  purchased. 
This  figure  is  proportional  to  gross  savings  on  these  quantities.  This 
measure  of  gross  savi  gs,  also  in  FY  1972  dollars,  was  used  in  the 
multiple  regression  analysis  conducted  by  the  APR078  analysts  in  their 
attempt  to  develop  a  predictive  modol.  These  results  on  gross  savings 
aro  shown  in  Table  6. 


Table  5 

ITEMS  IN  THE  APR078  SAMPLE  WITH  PARTLY  NETTED  SAVINGS  IN  DOLLARS 
ORDERED  BY  SAVINGS  AS  PERCENTAGE  OF  TOTAL  PROCUREMENT  COSTS 


Partly  Netted 
Savings/Losses  as 
Percent  of  Total 
Procurement  Costs 


Partly-Netted 
Savings/Losses 
(millions  of 
FY72  dollars) 


1. 

Shrike  Missile(a) 

51.0 

103.2 

2. 

PRC-77  Radio 

34.8 

52.6 

3. 

TOW  Launcher 

30.2 

83.5 

4. 

FAAR  TADDS 

18.2 

2.0 

5. 

FAAR  Radar 

16.6 

4.8 

6. 

Bullpup  AGM-12B  Missile(a) 

16.0 

38.3 

7. 

Dragon  Tracker 

12.0 

12.2 

8. 

TOW  Missile 

8.5 

61.3 

9. 

Shillelagh  Missile 

5.9 

18.3 

10. 

UPM-98  Test  Set 

3.0 

.08 

11. 

Dragon  Round 

2.7 

8.0 

12. 

ARC-131  Radio 

-  2.1 

-  .6 

13. 

Sidewinder  Missile 

AIM-9D/G  GCG(a) 

-  2.7 

-  1.9 

14. 

Standard  Missile(a) 

-  3.9 

-11.8 

15. 

Sidewinder  Missile 

AIM-9B  GCG(a) 

-  4.0 

-  6.7 

16. 

Mark  46  Torpedo (a) 

-13.2 

-52.9 

Arithmetic  Mean 

=  10.8 

Total  =  310.38 

SOURCE:  APR078 ,  Fig.  6-1,  p.  93. 

(a)  System  analysed  by  Kluge  and  Lieberraann,  TRI 


ITEMS  IN  THE  APR078  SAMPLE  WITH  AVERAGE  UNIT  PRICE  ACROSS 
ALL  POST-COMPETITION  PRODUCTION 


Item 


Gross 

Savings 

(percent) 


Post  -Coinpet  it  ion 
Unit  Price 
(1972  dollars) 


1. 

Shrike  Missile(a) 

52.8 

6309. 

2. 

PRC -77  Radio 

41.9 

589. 

3. 

FAAR  Radar 

39.5 

85805. 

4. 

TOW  Launcher 

34.7 

17702. 

5. 

FAAR  TADDS 

31.1 

2714. 

, 

Bullpup  AGM-12B  Missile(a) 

26.9 

3202. 

7. 

TOW  Missile 

12.4 

2114. 

8. 

Dragon  Tracker 

12.2 

5935. 

o  t 

UPM-98  Test  Set 

11.5 

8676. 

10. 

Shillelagh  Missile 

9.4 

2611. 

11. 

Dragon  Round 

2.9 

1828. 

12. 

Sidewinder  AIM-9D/G  GCG(a) 

.7 

6748, 

13. 

Standard  Missile(a) 

-  5.4 

46517. 

14, 

Sidewinder  AjiM-9B  GCG(a) 

-  5,5 

2151. 

15. 

ARC-131  Radio 

-16.1 

3714. 

16, 

Mark  46  Torpedo(a) 

-29.4 

35765, 

Arithmetic  Mean 

*  13-7 

SOURCE:  APKQ76.  Figure  4-2.  p.  69. 

(a)  Systems  analysed  by  Kluge  and  Lieburaann,  TK1. 
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The  authors,  Lovett  and  Norton,  are  somewhat  cautious  in  their 
conclusions  about  the  effects  of  price  competition: 

Much  has  been  written  about  competition  and  how  it  saves  the 
Government  money  when  introduced  into  weapons  systems  acquisition. 
Most  of  the  studies  claim  substantial  savings  through  reduced 
unit  prices  and  attribute  the  reduction  to  competition.  While  this 
report  supports  the  belief  that  savings  have  been  made  in  the  past 
and  will  continue  to  be  made  in  the  future  through  competition,  it 
recognizes  that  competition  does  not  always  result  in  a  savings 
to  the  Government,  and  that  when  savings  do  occur,  not  all  the 
price  reduction  creating  the  savings  is  due  solely  to  competition. 

A  portion  of  the*  price  reduction  is  due  to  contractor  learning. 

It  also  recognizes  the  expenses  incurred  by  the  Government  to 
obtain  a  competitive  environment  and  identifies  some  of  the  problems 
that  may  be  created  by  establishing  competition,  [p.  64] 

In  their  attempt  to  develop  a  predictive  model,  the  APRG78  authors 
checked  several  variables  (they  do  not  say  which  ones),  and  several 
different  transformations  (logarithm,  square,  square  root,  etc.).  They 
ended  up  with  a  model  in  which  the  outcome  variable  was  the  logarithm  of 
the  price  after  buy-out  competition,  and  the  predictor  variables  were 
the  logarithms  of  the  expected  sole-source  price  and  the  ratio  of  post- 
competitive-buy-out  quantity  to  total  program  quantity. 

Lovett  and  Norton  were  not,  we  believe,  sufficiently  cautious  in 
evaluating  the  adequacy  of  their  predictive  model.  They  suggested  the 
model  can  be  used  “to  forecast  a  competitive  bid  price11  (pp.  70,75]  and 
"to  determine  the  ratio  of  quantities  at  which  the  projected  (sole- 
source)  unit  price  equals  the  forecasted  (competitive)  unit  price,"  {pp, 
73,77]  The  authors  cautioned  that  the  “quantitative  results  derived 
(from  this  model)  must  be  viewed  in  light  of  the  qualitative  factors 
which  influence  potential  savings  due  to  competition,"  (p,  7?)  and  they 
presented  two  management  tools  for  dealing  with  the  qualitative  factors: 
a  "competition  screen1*  to  be  applied  before  the  model  is  used  fpp, 

6?1  and  a  “competition  index**  to  be  used  after  obtaining  positive 
results  from  the  quantitative  model  | pp .  77-SIJ,  They  recommended  that 
the  model  be  tested  “on  a  system  meeting  the  required  competition 
structure  to  verify  its  applicability  and  accuracy"  and,  if  verified* 
that  the  model  be  adopted  by  the  Comptroller  of  the  Amy. 
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The  Fourth  Study:  IDA79 

This  IDA  project,  partly  overlapping  the  period  of  our  own  study, 
was  conducted  by  Daly,  Gates,  and  Schuttinga  for  the  Office  of  the  Under 
Secretary  of  Defense  for  Research  and  Engineering  (Acquisition  Policy). 
Its  purpose  was  "to  examine  the  benefits  and  costs  of  utilizing  price 
competition  during  the  reprocurcment  phase  of  the  weapon  system 
acquisition  process  and  to  determine:  (i)  when  competition  should  bo 
considered  .  .  (ii)  how  long  multiple  sources  should  be  maintained  if 
competition  is  introduced;  and  (iii)  the  changes  in  policies  and 
practices  which  would  improve  the  use  of  competition."  [p.  S-l] 

The  IDA79  analysts,  in  estimating  gross  savings  and  in  attempting 
to  develop  a  predictive  model,  used  data  assembled  by  the  other  three 
studies.  After  discarding  items  for  which  there  were  fewer  than  two 
price-quantity  data  points  (p.  A-2|,  IUA79  was  left  with  31  itens,  as 
shown  in  Table  7.  Included  were  the  six  electronic  items  that  appeared 
in  both  the  ECOM72  and  1DA74  analyses;  the  three  missiles  included  in 
both  IDA74  and  APRQ7S;  four  more  missiles  and  the  MARK-46  torpodo  from 
APR07S;  seven  other  electronic  items  from  EC0M72;  and  ten  asserted 
systems  from  I0A74,  including  two  on  which  Zusman  ot  al,  collected  data 
but  discarded  from  their  'sample'  because  the  unit  prices  were  below 
51000  (the  Aerno  42-0750  voltage  regulator  and  the  Aerne  42-2028 
generator).  Thus  this  1BA79  'sample'  of  31  includes  more  low  unit-price 
items  and  more  electronic  items  than  either  the  IDA74  or  the  APK078 
studies,  As  shown  in  Pig,  2,  over  two-thirds  of  the  31  items  had  a 
cumulative  average  price  of  less  than  510. @90  at  the  end  of  sole-source 
production,  i.e,,  when  the  price,  especially  the  cumulative  average 
price,  was  still  generally  high. 

Although  the  IBA74  authors,  in  reporting  results  on  actual  savings, 
sometimes  refer  to  "net  savings'*  (in  thitir  Table  4,  for  instance),  the 
only  costs  of  competition  netted  out  are  "the  contract  costs  of  learning 
buys  for  the  second  source  and  split-award  competitions,**  {p.  &£) 

"Savings  are  calculated.**  we  are  told,  "fey  subtracting  the  actual  cost 
to  the  government  (contrast  price)  of  all  post  sole-source  production 
contracts  from  the  price  projected  on  the  basis  of  the  sole-source 
progress  curve  and  then  expressing  the  differe»ce  as  a  percentage  of  the 


ITEMS  IN  THE  IDA79  SAMPLE,  ORDERED  BY  GROSS  SAVINGS 


Item 

Gross  Savings  on 

All  Post -Competition 
Production 
(in  percent) 

1. 

CV-1543 

64.0 

2. 

TD-204 

62.1 

3. 

AN/APM-123 

61.2 

4. 

AN/GRC-103 

58.7 

5. 

MD-522 

58.6 

6. 

TD-352 

58.0 

7. 

USM-181 

56.0 

8. 

NK -980 /PI'S -5 

56.0 

9. 

ARC-54 

55.0 

10. 

Aerrto  42-0750 

54.8 

11. 

Aerno  60-6402 

49.4 

12. 

SPA -25 

48. 8 

13. 

TD-2Q2 

46.8 

14, 

HAWK  metal  motor  parts 

45.7 

15. 

TOW  Launehor 

44.2 

16, 

AN/GRC-106 

43.3 

17. 

PRT-4 

42.3 

16, 

TAIOS  Guidance  &  Control 

40,8 

19. 

TP- 660 

3«,3 

20, 

Bull pun  Missile 

31,7 

21, 

AN7AFX-72 

27.1 

22, 

r6G'20 

23.7 

'itn 

fe  V?  ‘ 

l‘KC-77 

20,5 

2*v 

Aern©  42-2928 

19,9 

25, 

Reekeve  Cluster  lotsb 

11.6 

26- 

TOW  Missile 

6.9 

27, 

AIM-98  Suidaits©  4  Control 

1,6 

26, 

Standard  Missile 

-  4,2 

29. 

AIM-90/6  Guidance  4  Csntrol 

*  4,6 

SO. 

Shillelagh  Missile 

-  8,0 

31. 

MS -46 

-as.e 

Arithmetic  Mean  “35,1 

tfWfUXt  1&A79,  Table  A- 1,  p.  A-4. 


‘OansMty&w*'  fwrwjp  (*fc*  ft  «n4  <>f  tofe-iawte  tar  eduction. 
Irwnfs*  •  10*3®,  Estate  4'  I,  p.  4MI 
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projected  sole-source  price."  [p.  62]  The  IDA79  analysts  used  cumulative 
average  prices  to  derive  the  sole-source  progress  or  learning  curve  and 
adjusted  to  constant  dollars,  although  they  do  not  report  the  year  to 
which  the  adjustment  was  made.  The  savings  estimates  were  not  discounted. 

IDA7S's  attempt  to  develop  a  predictive  model  differed  from  the 
other  three  studies.  The  outcome  variable  IDA79  wished  to  predict  .is  a 
measure  referred  to  as  "the  competitive  learning  curve  slope."  This 
measure  is  actually  the  slope,  on  a  log-log  grid,  of  the  line  running 
from  a  point  representing  the  cumulative ^average  price  at  the  end  of 
sole-source  production  to  a  point  representing  the  cumulative  average 
price  on  the  total  procurement  quantity.  It  seems  misleading  to  refer 
to  this  line  as  a  competitive  learning  curve:  it  reflects  post- 
competition  price  behavior  only  in  a. very  indirect  fashion  and  its 
relationship  to  post-competit ' on  production  costs  is  unknown. 

IUA79's  expectation  was  that  the  slope  of  this  "competitive 
learning  curve"  could  be  predicted,  on  the  basis  of  a  liwear  regression, 
from  the  slope  of  the  known  sole-source  learning  curve.  Sole-source 
slopes  were,  as  already  mentioned,  derived  from  cumulative-average- 
price/quantity  data  points.  IDA79  reported  that  when  "just  one  aberrant 
item  (the  SPA-25)"  was  omitted  from  the  regression,  the  outcome  variable 
("competitive  learning  curve  slope")  and  the  predictor  variable  (solfr- 
source  learnirg  curve  slooe)  were  uncorrelated.  [p,  A-5]  This  left 
IDA79  having  to  use  the  mean  of  the  measuro  referred  to  as  "competitive 
slope"  in  the  equation  which  is  supposed  to  predict  gross  savings  on 
competitive  buys.  The  equation  is  the  cumulative  average  variant  of  the 
standard  learning  curve  equation:  gross  savings  on  competitive 
quantities  are  "predicted"  as  a  function  of  the  ratio  of  total  quantity 
to  sole-source  quantity,  known  sole-sourco  slope  derived  from  cumulative 
average  prices,  ~nd  the  m«ao  (-.414)  of  "eompst it ivo  slopes"  from  this 
'sample'  of  31  items. 

IDA79  concluded--too  optimistically  in  our  view- -that  this  "savings 
forecasting  model  is  only  a  moderately  successful  predictor  of  actual 
savings."  [p.  A- 7] 

It  is  haid  to  tell  whether  I DA 79  is  actually  recommending  the  model 
as  a  useful  management'  tool.  Thu  model  itself  is  presented  in  IDA/V's 
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App.  A  and  is  used  in  App.  C  to  estimate  what  the  savings  would  have 
been  had  the  Improved  Hawk  been  competitively  reprocured.  It  is, 
however,  not  even  mentioned  in  App.  F,  where  stylized  examples  are  used 
"to  illustrate  the  sensitivity  to  various  assumptions  of  the  estimated 
savings  attributed  to  the  introduction  of  competition."  [p.  F-l]  Nor  is 
it  mentioned  in  the  text  of  the  report,  which  states: 


The  reduction  in  unit  cost  is  the  most  difficult  component  (of 
the  internal  rate  of  return)  to  forecast.  It  is  in  fact  like¬ 
ly  that  no  precise  and  stable  predictive  relationship  exists; 
there  are  so  many  dimensions  of  variation  surrounding  each 
procurement  (e.g.,  technology,  market  conditions)  that  each 
system  is  to  a  considerable  extent  unique. 

Experience  with  previous  systems  reveals  considerable  varia¬ 
tion  in  the  realized  gross  savings  in  uiiit  prices  after  com¬ 
petition.  (Findings  on  split-award  and  buy-out  competitions 
are  then  summarized. ) 

Based  upon  this  information,  reasonable,  yet  conservative  fig¬ 
ures  for  the  projection  of  post  competition  savings  are  10 
percent  for  split '•award  buys  and  20  percent  for  buy-outs. 

These  numbers  can,  of  course,  be  adjusted  to  incorporate  ill- 
formation  regarding  the  circumstances  surrounding  the  procure¬ 
ment  of  a  particular  system,  (pp.  83-84] 

Although  IDA7V  makes  much  of  the  need  to  discount  [Ch.  VII  and  pp, 
85-86),  that  is,  to  treat  price-competitive  reproeurement  as  an 
investment:  decision  in  which  the  time  stream  of  costs  and  benefits  is 
taken  into  account  by  computing  a  net  present  value  or  internal  rate  of 
return,  the  authors  ignore  the  effects  of  discounting  and  the 
consequences  of  netting  out  in-house  and  external  costs  in  their  summary 
advice  to  decisionmakers  quoted  above.  If  program  managers  (U  project 
gross  savings  of  10  percent  on  spUt-be.  «  and  20  percent  on  buy-outs,  as 
suggests,  (2)  take  into  account  the  costs  of  introducing 
competition,  and  (3)  estimate  the  net  present  value  or  internal  rate  of 
return,  they  are  mast  unlikely  to  teete  up  with  numbers  suggesting  that 
price-competitive  reprocuresent  is  a  worthwhile  investment. 

WH&T-CfeS  SK  LbAKSEB  ratM_TUK.SS. 

These  four  studies  contribute  to  our  understanding  of  the 
competitive  reprocureoent  process,  but  they  do  not  iwfth  the  passable 


exception  of  electronic  items)  provide  convincing  evidence  of  savings 
due  to  competitive  reprocurement,  nor  do  they  provide  reliable 
quantitative  tools  for  decisionmaking. 

Credibility  Problems  Associated  with  Savings  Estimates 

If  a  decisionmaker  is  aware  of  the  problems  that  may  arise  in 
transferring  a  complex  production  technology,  of  the  additional 
administrative  costs  that  will  be  incurred  in  setting  up  and  managing  a 
competitive  production  environment,  and  of  the  significance  of 
discounting,  he  is  likely  to  decide  that  competitive  reprocurement  is  a 
risky  investment  indeed.  And  the  doubts  acquisition  decisionmakers 
harbor  about  the  dollar  benefits  derivable  from  competitive 
reprocurement  are  likely  to  be  increased  rather  than  decreased  by  a 
careful  perusal  of  the  existing  quantitative  research. 

Savings  estimates  made  by  different  analysts  on  a  single  system 
often  vary  enormously.  Table  8  illustrates  this  phenomenon  in  the  cast 
of  the  Shillelagh  missile.  Different  savings  estimates  calculated  on 
the  Shillelagh  run  all  the  way  fi-om  79  percent  to  -14  percent.  Some  of 
the  reasons  for  this  extreme  variability  on  the  Shillelagh  and  on  other 
systems  are  (l)  some  analysts  have  calculated  savings  on  tha  first 
competitive  buy  while  others  have  estimated  savings  across  all  post- 
competition  production;  (2)  where  adjustments  for  sole-source  learning 
have  beau  wade,  different  methods  have  boon  used  to  estimato  tho  sole- 
source  curve;  and  (3)  some  analysts  have  netted  out  some  costs  while 
others  report  gross  savings.  Half  the  estiiMtos  iu  Table  8  (those  from 
p.  SB  in  IDA79)  were  in  fact  calculated  by  IDA79  to  show  the  sensitivity 
of  savings  estimates  to  (2)  and  (3).  If  these  five  estimates  that  are  a 
result  or  U>A?9's  sensitivity  analysis  are  omitted,  results  still  range 
from  79  percent  to  -8  percent. 

Some  of  the  variation  in  results  shown  on  Table  8  stem  from 
different  definitions  of  savings,  but  differences  in  basic  data 
interpretation  also  exist.  The  result  is  that  a  program  manager  looking 
for  evidence  of  savings  iu  previous  programs  can  find  a  bewildering 
variety  of  answers,  leading  to  serious  data  credibility  problems  and 
consequent  disincentives  to  the  use  of  competition. 


DIVERSITY  OK  SAVINGS  ESTIMATES  ON  THE  SHILLELAGH 


Percent 

Savings 

Estimate 

1 

1 

|  Source 

1 

i 

i 

1 

|  Method 

1 

79 

Yuspeh  (1976), 
p.  ill. 

Lowest  price  [Martin's]  on  second  split-buy 
compared  with  last  sole-source  price,  in  1970 
dollars.  No  adjustment  for  sole-source  learning. 

32 

Carter  (1974), 

p.  120 

Solo-source  learning  curve  projected  from  data  in 
Yuspeh  (1973),  and  savings  computed  as  "the 
difference  between  the  competitive  bids  of  the 
developer  and  the  second  producer"  [Martin], 
in  1970  dollars. 

OO 

IDA79,  p.  58(a) 

Savings  on  competitive  bi.y-out,  recurring  costs 
only,  presumably  in  1972  dollars.  Sole-source 
learning  curve  exponent  of  -0.233  reported  in 

APRQ76  used  to  project  sole-source  price. 

6 

AFR068,  p.  33 

Savings  on  all  post -competition  production  minus 
8746,000  in  technology  transfer  costs  as  a 
percentage  of  total  procurement  costs,  in  1972 
dollars.  Sole-source  learning  curve  exponent  of 
-0.233  derived  from  cost  data  for  first  two  sole- 
source  buys- 

9 

IDA79,  p.  38(a) 
APK07S.  p.  66 

Savings  on  all  post-competition  production,  re. 
eurring  eosts  only,  in  1972  dollars.  Sole-source 
learning  curve  exponent  of  -0.233  from  APR078. 

0 

ll)A?4,  p.  36 

Savings  on  first  competitive  split-buy,  in  1970 
dollars.  Sole-source  learning  curve  exponent  of 
-0.393  derived  Ire*  price  data. 

•l 

IBA79,  p.  3S(a. 

Savings  on  first  competitive  split -bay,  presumably 
in  1972  dollars,  using  the  APRQ78  exponent  of 
-0.233  for  the  sele-seuree  learning  curve. 

■4 

?UA79,  p,  m»i 

Saving*  on  all  pest -competition  production,  re¬ 
curring  costs  only,  using  a  learning  curve  exponent 
(value  net  given)  derived  by  I8A79  turn  A?K@73 
data.  Presumably  197!  dollars. 

*8 

IBA'I.  p.  A-A 

Savings  on  all  pe&t-eospetfti&n  product  ion,  re¬ 
curring  tests  only,  using  cumulative  average  price 
learning  curve  exponent  es treated  at  *6,390. 
Constant  dollars,  year  nut  given. 

•Is 

10A79,  *>.  38(0) 

Savings  us  first  campatttive  split-buy  using 
learning  curve  exponent  (the  value  is  n&t  given*' 
derived  by  tDA79  free  Affc§*8  data,  rresvambly 

1972  dollars. 

(a)  These  estimates  tree*  calculated  by  IBA79  t©  demonstrate  the  sensi- 
livUy  of  savings  estimates  “to  different  metb<«fx  of  estimating  progress 
curves  an*  to  the  instvsien  of  different  tests.1*  l&A’b  also  calculated  ether 
estimates  oo  the  Shillelagh  «»et  included  here  anil  equivalent  series  of 
estiaates  for  alt  the  other  system  In  the  M*(S?8  study  See  i8*N,  JNS|. 
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Another  indication  of  these  credibility  problems  is  the  lack  of 
ordinal  agreement  among  studies  analyzing  the  same  subsets  of  items.  As 
Table  9  indicates,  there  is  disagreement  in  the  rank-orderings  by 
percentage  gross  savings  {on  all  post-competition  production)  reported 
by  APR07S  and  IDA79  on  the  subset  of  items  analyzed  by  both  studies. 

This  lack  of  even  ordinal  agreement  between  the  findings  from  two 
studies  using  the  same  basic  data  is  at  least  as  worrisome  as  the 
disagreement  among  studies  on  a  single  system.  The  major  stated 
difference  between  APk078's  method  of  estimating  gross  savings  and 
IDA79's  lies  in  the  derivation  of  sole-source  learning  curves:  APRQ78 
used  unit  production  costs,  whereas  IDA79  used  cumulative  average 
prices.  Savings  estimates  are,  as  IDA79  pointed  out  (pp.  57-79),  very 
sensitive  to  the  method  of  deriving  the  sole-source  learning  curve;  it 
appears  that  even  rank-orderings  of  savings  are  quite  radically  effected. 

Taking  into  Account  the  Time  Value  of  Honey 

The  decision  to  reproeure  competitively  should  be,  as  IDA79  argues, 
looked  upon  as  an  investment  decision:  costs  are  incurred  in  the  present 
with  the  expectation  of  obtaining  savings  in  the  future.  Because  the 

Table  9 

RANK  ORDERING  BY  POST-COMPETITION  GROSS  SAVINGS  REPORTED  BY  APRQ7S 
AND  IDA79  OK  ITEMS  INCLUDED  IN  BOTH  ‘ SAMPLES 


"  APloift  “  imi 

It<*a  APE9?B  Rank  IDA7P  Rank  Svess  Gress 

within  within  Savings  Savings 

Subset  Subset  £8  thousands)  £S  thousands) 


PRC-7?  Radio 

1 

5 

41.9 

26.5 

TOW  Lanneher 

2 

1 

34,7 

4w .  i 

Bull pup  AGM-12B  Missile 

3 

2 

2b,  9 

51 .  ? 

TOW  Missile 

/• 

«* 

4 

12.4 

6.9 

Shillelagh  Missile 

5 

8 

9.4 

-  6.0 

Sidewinder  AtMi-SD/G  GCC 

6 

7 

,*4 

.  t 

-  4.8 

Standard  Missile 

7 

0 

-  5.4 

“  4,2 

Sidewinder  AIM-98  CCS 

6 

5 

-  5,5 

1.6 

Mark  4t»  Torpedo 

9 

9 

-29.4 

-25,0 

savings  will  begin  to  accrue  several  years  after  the  initial  investment 
in  competition  is  made,  it  is  important  to  take  into  account  the 
opportunity  cost  of  Government  funds.  Savings  realized  from  competitive 
reprocurement  will,  when  discounted  at  even  the  10  percent  rate 
suggested  by  0MB  (let  alone  at  the  higher  rates  that  would  reflect  the 
perceived  scarcity  of  front-end  funds),  look  much  less  impressive  than 
undiscounted  savings. 

Among  the  studies  we  are  familiar  with,  IDA79  is  the  only  one  to 
have  examined  the  effects  of  discounting  and  then  for  only  two  systems: 
the  Sparrow  AIM-7F  Guidance  and  Control  System  and  the  TOW  Missile.  The 
only  obvious  omission  from  !DA79's  estimates  are  the  additional  in-house 
costs  of  introducing  competition. * 3  An  internal  rate  of  return  of  12 
percent,  computed  from  data  collected  by  IDA79,  is  reported  on  the 
Sparrow  AIM-7F  Guidance  and  Control  System  [p.  B-9],  a  system  that  was 
competed  for  non-price  reasons.  On  the  TOW  Missile,  IDA79  reported  an 
estimated  internal  rate  of  return  of  24.2  percent,  computed  from  data 
collected  by  APR078.  Using  the  10  percent  discount  rate  recommended  by 
0MB,  the  net  present  value  calculated  by  IDA79  for  the  TOW  Missile  was 
S19  million  in  1972  dollars,  (pp.  75-76J 

By  considering  the  APRQ/6  results  on  gross  savings,  we  can  get  a 
rough  idea  of  the  effect  of  discounting  when  a  portfolio  of  relatively 
complex  systems,  especially  missiles,  is  being  reproeured.  The  mean  of 
gross  savings  on  all  post -competition  production  for  APRQ78's  'sample' 
of  16  items,  mainly  Army  and  Navy  missiles,  was  13,7  percent.  Thus  if 
the  in-house  and  external  costs  of  introducing  competition  were  taken 
into  account  and  if  costs  and  savings  we so  duly  discounted,  the  net 
present  value  of  investments  in  competition  for  this  portfolio  of 
systems  would  be  negative --a  fairly  s Usable  loss,  in  fact.  Whether  this 
loss  should  be  attributed  to  competition  or  to  badly  designed  and 

iS  The  Sparrow  estimate  dees  take  into  account  the  costs  of  a 
contract  with  the  Naval  Weapons  Center  at  China  Lake,  While  these  costs 
are  technically  in -house  costs,  and  are  so  listed  by  1BA79,  they  covered 
the  preparation  of  a  TOP  and  technical  support  to  the  second  source. 

Thus  they  are  associated  with  the  technology  transfer  process  that  would 
normally  be  external,  rather  t'.ian  with  the  additional  administrative 
load  placed  on  Program  Offices  managing  competitive  environments. 


implemented  acquisition  strategies  (for  instance,  the  failure  of  a 
program  to  prep' re  well  in  advance  for  the  introduction  of  competiton  in 
the  production  phase)  remains  an  open  question  at  this  stage  of  our 
analysis. 


Predicting  Price  Reductions 

Only  two  of  the  four  studies--APR078  and  IDA79--claimed  to  have 
some  success  in  developing  a  model  that  decisionmakers  could  use  to 
predict  price  reductions  attainable  from  competitive  reprocurement. 
APR078  offered  as  a  predictive  model  an  equation  in  which  the  outcome 
variable  is  the  logarithm  of  the  price  after  buy-out  competition,  the 
predictor  variables  are  the  logarithms  of  the  expected  sole-source 
price,  and  the  ratio  of  the  post-competitive-buy-out  quantity  to  the 
total  quantity.  The  choice  of  average  post-competition  unit -price  as 
the  variable  to  be  predicted  was  not,  unfortunately,  a  wise  choice.  A 
better  choice  would  have  been  the  difference  between  the  expected  sole- 
source  price(s),  itself  a  prediction,  and  price(s)  under  competitive 
conditions.  To  go  after  post-competition  price  directly,  as  APR07S  did, 
means  that  expected  sole-source  price  is  used  as  a  predictor  variable. 
The  correlation  between  expected  sole-source  price  and  actual  post¬ 
competition  price  is  so  high  (.979  between  the  logarithms  of  the  two 
prices  in  the  APRQ7S  study)14  that  it  swamps  the  effect  of  other 
variables. 

The  APR078  authors  stated  that  their  model  should  be  tested  "on  a 
system  meeting  (certain  qualitative  criteria)  to  verify  its 
applicability  and  aeeuracy"  before  being  adopted  by  the  Comptroller  of 
the  Army  as  "approved  procedure  for  independent ly  (i.e.,  independent  of 
the  program  managor)  forecasting  savings  expoetod  from  competition."  (p. 
96)  Testing  the  model  on  only  one  additional  system,  as  APK078 
recommended ,  would  not  oven  come  close  to  validating  it.11 

~  14  APR078  actually  reports  the  correlation  as  being  1.00.  [p.  70) 
Our  recalculation  was  0.979. 

14  See  Hosteller  and  Wallace  (1964)  for  a  good  case  study  of  how 
to  validate  models  on  additional  data. 
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The  IDA79  approach  was  interesting  and  worth  exploring  but,  given  the 
lack  of  correlation  between  the  sole-source  slope  and  the  "competitive 
learning  curve  slope,"  it  turned  out  to  be  a  blind  alley.  IDA79's  fallback 
position,  using  the  mean  of  the  mea'ure  referred  to  as  the  "competitive 
learning  curve  slope"  to  predict  gross  savings,  has  all  the  weaknesses  of 
any  attempt  to  predict  from  a  mean.  In  addition,  the  way  the  "competitive 
learning  curve  slope"  is  defined  reflects  the  one-time  effects  of 
competition  more  than  it  reflects  learning  in  the  post-competition  phase. 
IDA74  estimates  of  post-competition  learning  curve?  from  post -competition 
price  behavior  show  them  to  have  shallow  slopes,  whereas  the  mean  of  the 
measures  IDA79  refers  to  as  "competitive  learning  curve  slopes"  is  very 
steep--75  percent. 

IDA79’s  predictive  model  is,  as  previously  mentioned,  a  variant  of 
the  standard  learning-curve  equation.  It  is  not  a  result  of  IDA79's 
regression  analysis.  It  is,  therefore,  misleading  when  the  authors,  in 
discussing  the  implications  of  the  equation,  state:  "As  one  would 
intuitively  expect,  the  greater  the  quantity  competed,  the  greater  the 
expected  percent  savings."  [p.  A-5]  The  effect  of  the  quantity  ratio  in 
IDA79's  equation  is  determined  solely  by  the  assumptions  of  learning- 
curve  theory;  the  quantity  ratio  in  this  equation  must  behave  exactly  as 
those  assumptions  say  it  should  behave,  i.a.,  with  utter  disregard  for 
the  data.  lDA79's  equation  sets  the  exponent  of  the  "competitive 
learning  curve"  equal  to  -.414,  a  slope  of  75  percent,  for  all  time. 

That  is  a  rapid  rate  of  learning.  Few  sole-source  slopes  will  be  that 
steep,  so  it  is  little  wonder  that  the  greater  the  quantity  assumed  to 
be  produced  under  a  75  percent  curve,  the  greater  the  gross  savings. 

These  two  studies  (APRQ78  and  IDA79)  which  claim  to  have  achieved 
some  success  in  developing  a  model  to  prodict  price  reductions  from 
competitive  reprocurement,  placed  heavy  reliance  on  production-cost 
learning  curve  theory.  They  accepted  tho  conventional  wisdom,  as  do  tho 
Defense  Acquisition  Regulations  (DARs),  that  the  greater  the  post- 
competition  quantity  relative  to  tho  sole-sourco  quantity  tho  greater 
tho  price  reductions  from  competition.  Yet  the  multiple  regression 
results  reported  by  the  two  earlior  studies,  KCOM72  and  IDA74,  show  tho 
rovorse:  tho  greater  the  post -competition  quantity  relative  to  the 
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sole-source  quantity,  the  smaller  the  unit-price  reductions  on  the  first 
competitive  buy.  EC0M72  and  IDA74  results  also  indicate  that  the  effect 
of  the  quantity  ratio  is  minor,  i.e.,  the  coefficient  of  the  quantity 
ratio  in  the  regression  equations  is  (relatively)  small.  The  J.DA79 
results  provide  no  empirical  information  about  the  effect  of  the 
quantity  ratio.  The  APR078  results  do:  they  are  consistent  with  the 
ECQM72  and  IDA74  results  in  showing  a  small  coefficient  for  the  quantity 
ratio  and  inconsistent  in  showing  a  positive  correlation  between  the 
ratio  of  post -competition  to  sole-source  quantities  and  unit-price 
reduction. 

This  one  positive  correlation  seems  to  suggest  that  production-cost 
learning  curve  theory  is  not  wholly  irrelevant  when  it  comes  to 
predicting  price  reductions,  but  all  the  other  findings  indicate  that  it 
has  not  been  helpful  in  explaining  the  results  of  competitive 
reprocurement.  It  may  even  have  been  a  hindrance.  IDA74  sounded  a 
warning  about  production-cost  learning  curve  theory.  The  authors 
pointed  out  that  learning  curve  theory  leads  one  to  expect  that  the 
original  vendor,  the  sole-source  producer,  should  be  able  to  win  a 
competitive  production  contract  easily  because  of  the  learning  that  has 
occurred  during  the  initial  period  of  sole-source  production.  This  is 
rot  what  happened,  however,  in  the  reprccurements  examined  by  IDA74. 

When  an  item  was  repvocured  competitively,  the  winning  bidder  in  a 
majority  of  cases  was  not  the  original  vendor.  We  were  able  to 
replicato  this  finding  from  1DA74  using  the  APRQ78  data. 

Learning  effects  presumably  need  to  be  taken  into  account  in  any 
model  of  competitive  roprocuremont .  but  how  that  should  be  dona  remains 
an  upon  question.  And  a  more  important  question  is:  What  other 
variables  should  be  taken  into  account  in  order  to  predict  the  price 
reductions  to  be  expected  from  competition? 

Current  understanding  of  the  competitive  reprocurement  process  is 
meager.  It  would,  for  example,  be  an  understatement  to  say  that  the 
determinants  of  post-competition  price  differences  have  not  yet  been 
identified.  We  were  unable  to  discover  a  relatively  complete  list  of 
even  the  potential  determinants. 
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From  these  four  studies,  we  do  have  some  quantitative  information 
on  several  proposed  determinants:  the  ratio  of  post-competition  to  pre¬ 
competition  quantities,  lead-time  ratios,  delivery-schedule  ratios,  the 
exponent  of  the  sole-source  learning  curve,  split-buys  versus  buy-outs, 
and  the  number  of  bidders.  There  is  some  additional  quantitative 
information  on  these  and  one  or  two  other  variables  in  other  studies  we 
are  familiar  with.16  Some  studies  also  provide,  between  the  lines  as 
it  were,  more  qualitative  information  on  several  other  potential 
predictor  variables:  complexity  of  the  item,  technological  innovation 
(both  rate  and  type)  in  the  relevant  industry,  opportunities  to  market 
similar  systems  in  the  private  sector,  management  of  the  technology 
transfer  process,  type  of  sole-source  contract,  etc. 

Price  differences  (they  are  not  always  reductions)  following  the 
introduction  of  competition  appear  to  be  determined  by  many  factors 
interacting  in  a  complex  fashion.  To  develop  a  model  that  can  explain 
and  predict  these  differences  may  require  a  thorough  search  through  a 
varied  literature:  work  on  technology  transfer,  technological 
innovation,  defense  industry  cost  management  and  pricing  strategies, 
etc .  We  have  not  undertaken  this  search,  nor  have  we  attempted  any 
secondary  statistical  analyses  of  the  data  assembled  by  hCUH,  APR 0  and 
IDA.1’ 


SUMMARY 

The  existing  body  of  analysis  has  not  provided  an  adequate  set 
management  tools  for  estimating  either  the  benefits  or  the  costs  ©£ 
competitive  reprocurement.  Furthermore,  it  indicates  that  much  of  the 

~*™1*  dee,  for  instance,  Brannon  ISA's  |l^ntrej4iC|»R  fey 

Sates  et  al.  Neat.e  and  Surges*  CI9T61;  and  Selinsky  hue 

to  various  proprietary  and  political  concerns ,  seme,  possibly  many,  sty* 
dies  that  have  been  conducted  for  Program  Offices,  for  0?ti,  Y©r  defense 
contractors,  etc.  are  inaccessible.  Perhaps  if  such  studies  were  more 
generally  available,  more  progress  could  be  made  in  understanding  com¬ 
petitive  reproturement  quantitatively, 

*  *  If  these  data  are  to  be  re-worked,  they  should  be  completely 
re-sforfeed.  This  woo to  involve  selfacting  additional  information;  at  the 
minimise,  additional  in format  ion  is  needed  to  clear  up  the  many  incon¬ 
sistencies  between  the  information  reported  on  a  particular  program  by 
different  studies. 
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convontional  wisdom  about  competitive  roprocureraent  rests  on  shaky 
foundations,  and  that  we  may  know  less  about  competitive  reprocurement 
than  we  thought  we  did.  However,  some  conclusions  seem  warranted: 

o  Savings  on  electronic  items  reprocured  competitively  by  the 
Army  have  been  substantial,  but  we  do  not  yet  know  to  what 
extent  these  savings  are  dependent  rn  (a)  technological 
innovations  in  tho  electronics  industry  confined  to  a 
particular  time  period,  (b)  ECOM's  or  the  Army’s  effective 
management  of  electronics  acquisitions,  (c)  the  relative  ease 
of  tvanferring  the  necessary  technology  from  one  electronics 
firm  to  another,  (d)  the  existence  of  a  commercial  market  for 
similar  items  which  motivates  producers  to  invest  their  own 
funds  to  ensure  that  the  technology  ti^isfer  process  is 
successful,  or  any  of  several  other  explanations. 

o  One  of  tho  major  decisions  in  a  competitive  rcprocurement  is 
when  in  the  production  tun  to  introduce  the  second  source.  If 
it  is  done  too  early,  there  is  an  inadequate  experience  base 
with  the  sole  source:  his  costs  are  not  well  established,  and 
the  design  itself  may  still  be  urdergoing  some  evolution-  If 
the  second  source  is  introduced  too  late,  on  the  other  hand, 
opportunities  for  savings  may  have  been  forgone-  The  BAR  may 
sot  be  promulgating  the  most  economically  beneficial  advice  in 
suggesting  that  quantities  procured  under  an  initial  sole- 
source  production  contract  be  kept  to  a  minimum  so  that 
competitive  reprocurement  can  be  introduced  as  early  as 
practicable-  Hinimiaiag  sole-source  quantities  and  maximizing 
post -competition  quantities  does  net  appear  to  generate  the 
largest  post -competition  price  reductions-  Without  verifying 
and  re-analysing  the  existing  data,  however,  we  cannot  say  what 
should  be  done  instead-  And  we  cannot  even  be  sure  that  a  re- 
analysis  or  existing  data  would  tell  us,  within  any  acceptable 

*"  bee  Gates  et  ai .  Electronics -SC ,  ?os  a  discussion  of 

some  of  these  issues. 
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confidence  intervals,  what  we  would  need  to  know.  It  may  be 
necessary  to  collect  data  on  additional  instances  of 
competitive  reprocureraent  to  understand  the  relationship 
between  quantity  ratios  and  price  reductions. 

While  production  costs  tend  to  decrease  with  increasing 
quantities  produced,  suggesting  that  the  original  producer 
should  be  able  to  win  a  competitive  award  hands  down,  we  find 
that  the  original  producer  often  loses  when  an  item  is 
reprocured  competitively.  But  we  do  not  know  why.  IDA74 
presents  some  convincing  evidence  suggesting  that  ASPR  profit 
weights  for  solo-source  contracts  will  motivate  a  profit- 
maximizing  producer  to  set  up  a  production  process  that  is 
inefficient.  Then  when  this  producer  faces  competition  from  a 
second  source,  he  stems  unable  to  shift  to  a  more  efficient 
product  ion  process . 

It  is  not  clear  whether  competitive  reproeurement  pays  off  as  a 
financial  investment  on  systems  as  complex  as  missiles,  because 
there  is  as  yet  no  evidence  that  internal  rates  of  return  are 
high  enough  to  justify  the  drain  art  front-end  funds. 

Here  might  be  learned  from  a  re-analysis  ©£  the  data  already 
assembled,  assuming  that  at  least  the  major  inconsistencies  and 
omissions  in  these  data  could  be  rectified;  it  is  not  possible 
at  this  point  to  estimate  Haw  much  more,  because  data  problems 
are  severe,  Bata  on  competitive  reproeuremeots  are  difficult 
to  retrieve;  and  adequate,  complete,  and  consistent  date  are 
almost  non-existent,  according  to  those  who  have  assembled  the 
current  data  base.  If  data  collection,  storage,  an1*  retrieval 
system*  remain  as  they  are,  our  quantitative  understanding  of 
the  competitive  reproeurement  process  is  likely  to  remain  weak 
into  the  If 90S, 
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IV.  CONCLUSIONS  AND  RECOMMENDATIONS 


As  a  system  moves  through  the  acquisition  t/cle,  the  application  of 
competition  changes  in  two  major  ways:  the  benefits  sought  from 
competition  change,  as  does  the  perceived  ratio  of  costs  incurred  to 
benefits  obtained.  Thus  an  analysis  of  the  role  of  competition  in 
weapon  system  acquisition  must  examine  individually  at  least  three 
distinct  acquisition  phases.  Prior  to  the  beginning  of  full-scale 
development,  the  benefits  sought  from  competition  are  a  complex  blend  of 
desires  to  improve  the  design  concept,  reduce  performance  and  schedule 
risks,  and  minimize  cost.  In  this  phase  the  dollar  investments  needed 
to  establish  and  maintain  competition  are  rather  small,  although  . 
additional  management  burdens  are  imposed  on  the  Program  Office. 
Conversely,  the  benefits  of  competition  are  widely  perceived  as  being 
valuable.  This  perception  appears  to  be  rooted  in  "conventional  wisdom" 
rather  than  the  results  of  any  quantitative  comparisons,  and  in  fast 
quantitative  analysis  is  virtually  impossible  due  to  the  multiple 
objectives  and  the  difficulties  of  comparing  outcomes  in  competitive  and 
non-competitive  programs.  However,  the  perception  of  benefits  is  widely 
and  strongly  held,  and  competition  is  the  narm  unde:,'  these 
circumstances. 

As  the  design  move*  into  full-scale  hardware  development,  she 
ehoiees  of  design  concept  are  narrowed  (usually  te  a  single  design)  and 
the  anticipated  benefits  at  competition  shaft  toward  reduction  at  risks 
and  easts.  In  this  phase*  however ,  the  east  of  maintaining  multiple 
sources,  rapidly  escalates  and  the  belief  in  equivalent  benefits 
dwindles,  Again  the  perception  of  benefits  is  highly  subjective,  with 
virtually  m  basis  in  quantitative  comparisons  between  sole-source  and 
competitive  development  programs.  Competition  is  rare  at  this  stage  of 
weapon  system  acquisition. 

Competition  in  the  production  phase  is  usually  viewed  as  a  weans 
of  reducing  cost  the  buyer  f  although  other  benefit*  are  possible 
and  sometimes  .sought).  As  in  tie-  full  “scale,  development  phase,  the 
investment  needed  »s  introduce  s.u  rresjuontly  sizable, 

ikwevo?,  in  this  phase  we  encounter  for  tine  first  tine  an  opportunity  to 
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collect  quantitative  evidence  on  the  benefits  of  introducing  competition. 

A  price  pattern  established  by  an  initial,  sole-source  production  program 
can  be  compared  with  actual  bid  prices  when  competitive  sources  are  brought 
into  the  program.  Unfortunately,  due  to  inadequacies  in  both  theory  and 
data,  such  comparisons  have  not  yet  produced  a  methodology  enabling  a 
program  manager  to  confidently  predict  post-competition  prices  and  thus 
potential  savings.  Or.  the  other  hand,  at  least  some  of  the  costs  of 
qualifying  a  second  source  (transfer  of  technology,  production  and  test  of 
iuitial  qualification  quantities,  etc.)  are  quite  apparent,  and  the  manager 
finds  in  the  DARs  and  the  policy  directives  only  narrow  justifications  for 
competition.  The  combined  result  is  that  the  introduction  of  competition 
into  an  ongoing  production  program  appears  risky  to  a  program  manager. 

Thus  it  is  not  the  common  practice,  especially  in  the  more  complex  "major** 
system  acquisition  programs.  The  diversity  of  costs,  desired  b  its.  and 
levels  of  perceived  uncertainty  regarding  the  introduction  of  %  ition 
during  these  three  phases  of  acquisition  illustrate  the  difficulty  of 
changing  the  amount  and  utility  of  competition. 

Although  considerable  research  m  competition  has  been  performed, 
reliable  guidelines  for  acquisition  managers  have  not  yet  been 
developed.  Existing  research  provides  neither  quantitative  nor 
qualitative  guidance  for  designing  price -competitive  reprocureeent 
strategies— one  of  the  simplest,  and  certainly  the  most  quantifiable, 
uses  of  competition,  ftethodelegies  should  be  developed  to  forecast  and 
evaluate  the  casts  and  benefits  of  various  competitive  strategies —net 
just  the  effect  es  price  of  competition  in  the  reprecurement  phase. 

Lacking  such  methodologies ,  program  personnel  are  unable  effectively  to 
justify  competition  expenditures  and  SeS  is  unable  to  de^lep  criteria 
for  selective  application  of  cespes4si«a.  Further,  without  knowing  hew 
re  evaluate  competition  in  the  context  or  acquisition,  decisiee&akers 
will  tend  to  focus  on  the  degree 'to  which  competition  is  or  is  not 
applied,  instead  of  hew  ouch  it  kns  or  has  not  increased  the 
effectiveness  of  systems  acquisition.  The  development  of  such 
methodologies,  however,  will  he  neither  simple  nor  rapid,  given  the 
difficulties  of  retrieving  adequate  information  about  past  program* . 
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One  simple  but  valuable  step  would  be  to  ensure  that  adequate 
records  are  retained  on  current  competitive  acquisition  actions  so  that 
future  analysis  can  benefit  from  an  adequate  data  base.  Such  a  data 
base,  and  the  subsequent  analysis,  should  focus  on  a  broad' set  of 
characteristics  of  individual  weapon  systems  as  determinants  of  contract 
outcomes,  as  opposed  to  only  learning  curves  and  cost  relationships. 

The  following  are  examples  of  the  kinds  of  information  that  would  be 
useful  to  the  analyst  but  that  are  not  now  collected  in  any  systematic 
way: 

o  The  degree  of  technological  sophistication  involved  in  the 
system,  and  the  technical  advance  it  represents  over  the 
previous-generation  system. 

o  The  program  environment,  in  terms  of  the  various  risks 

perceived,  the  urgency  of  development,  the  number  of  competing 
firms  involved  and  their  relativo  capabilities, 
o  The  business  approach  used  (the  organization  of  the  Service  and 
contractor  management  teams,  types  of  contracts  used), 
o  The  expected  size  of  the  production  run,  both  formally  stated 
and  informally  expected  by  the  participants, 
o  The  business  posture  of  the  competing  firms  (business  base, 
availability  of  key  personnel  to  work  on  the  subject  project, 
existence  of  special  facilities  or  other  elements  that  might 
give  it  a  competitive  edge,  etc.), 
o  The  extent  and  type  of  competition  that  existed  before  the 
beginning  of  F.SD,  or  the  beginning  of  production, 
o  The  quality  of  the  resulting  product,  and  the  reasons  for  any 
quality  or  other  major  shortfalls. 

The  role  of  the  contract  should  also  be  considered  in  more  detail. 
Even  when  contracts  are  classified  as  being  of  the  same  typo,  many 
aspects  may  vary  widely,  such  as  escalation  clauses,  penalties,  award 
fees,  payment  ceilings,  and  so  forth. 

Finally,  the  underlying  theory  of  how  compotion  should  function  in 
weapon  system  acquisition  appears  to  be  inadequately  developed. 


Numerous  important  questions  have  not  been  adequately  examined, 
including: 


o  In  what  circumstances  does  competition  lead  to  cost  reductions 
in  production,  or  profit  reductions,  or  some  combination  of  the 
two? 

o  Does  competition  influence  a  firm's  efficiency  by  inducing  it 
to  invest  in  capital  equipment,  manufacturing  technology,  or 
prodr ~.t  development?  Under  what  circumstances? 
o  How  does  the  firm's  general  business  situation  and  alternative 
investment  strategies  affect  the  impact  of  competition? 

Research  on  the  development  and  refinement  of  such  elements  of  a  theory 
would  be  extremely  valuable  if  it  could  provide  a  solid  framework  to 
guide  the  collection  of  data  and  the  design  of  future  data  analysis. 
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This  report  exaaines  aethods  for  iaprovioa 
the  effectiveness  of  competition  in  the 
acauisition  of  malor  weapon  systems.  Use 
of  competition  at  the  concept  desiqn  staqe 
is  known  to  be  inexpensive  relative  to 
benefits.  However,  competition  is  rare 
darinq  procurement  when  the  expense  is 
immediate  and  any  savinqs  are  delayed, 
Existinq  analysis  provides  no  means  for 
evaluatino  competitive  reprocurement. 
Savinqs  on  competitively  reprocured 
electronic  items  have  been  substantial  but 
the  factors  explaininq  this  are  unknown. 

The  best  point  in  production  at  which  to 
introduce  a  second  source  cannot  now  be 
determined.  Also  unclear  is  whether 
competitive  reprocurement  pays  off  for 
complex  systems.  Because  price-competitive 
reprocurement  strateqies  are  one  of  the 
simplest  and  most  quantifiable  uses  of 
competition,  this  study  recommends  improved 
data  collection  on  current  competitive 
acquisition,  and  further  theoretical  study 
of  the  function  of  competition  in  such 
acquisition.  ^ 
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